pg.ddx.io pgsql-general@postgresql.org mailing list archive
help / color / mirror / Atom feedSupport logical replication of DDLs
449+ messages / 37 participants
[nested] [flat]
* Support logical replication of DDLs
@ 2022-02-21 15:53 Zheng Li <zhengli10@gmail.com>
0 siblings, 3 replies; 449+ messages in thread
From: Zheng Li @ 2022-02-21 15:53 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org
Hello,
One of the most frequently requested improvements from our customers
is to reduce downtime associated with software updates (both major and
minor versions). To do this, we have reviewed potential contributions to
improving logical replication.
I’m working on a patch to support logical replication of data
definition language statements (DDLs). This is a useful feature when a
database in logical replication has lots of tables, functions and
other objects that change over time, such as in online cross major
version upgrade.
I put together a prototype that replicates DDLs using the generic
messages for logical decoding. The idea is to log the candidate DDL
string in ProcessUtilitySlow() using LogLogicalMessge() with a new
flag in WAL record type xl_logical_message indicating it’s a DDL
message. The xl_logical_message record is decoded and sent to the
subscriber via pgoutput. The logical replication worker process is
dispatched for this new DDL message type and executes the command
accordingly.
However, there are still many edge cases to sort out because not every
DDL statement can/should be replicated. Some of these include:
1. DDL involving multiple tables where only some tables are replicated, e.g.
DROP TABLE replicated_foo, notreplicated_bar;
This statement will fail on the subscriber and block logical
replication. It can be detected and filtered on the publisher.
2. Any DDL that calls a volatile function, such as NOW() or RAND(), is
likely to generate a different value on each replica. It is possible
to work around these issues—for example, the publisher can replace any
volatile function calls with a fixed return value when the statement
is logged so that the subscribers all get the same value. We will have
to consider some other cases.
3. CREATE TABLE AS and SELECT INTO, For example:
CREATE TABLE foo AS
SELECT field_1, field_2 FROM bar;
There are a few issues that can occur here. For one, it’s possible
that table bar doesn't exist on the subscriber. Even if “bar” does
exist, it may not be fully up-to-date with the publisher, which would
cause a data mismatch on “foo” between the publisher and subscriber.
4. Statements that have nondeterministic side effects (e.g., as caused
by triggers, stored procedures, user-defined functions) may result in
different side effects occurring on each subscriber.
Whether a DDL should be replicated also depends on what granularity do
we define DDL replication. For example, we can define DDL replication
on these levels:
1. Database level
Allows all DDLs for a database to be replicated except for certain
edge cases (refer to the edge cases mentioned above).
This is likely a major use case, such as in online major version upgrade.
2. Table level
Allows DDLs on the published tables to be replicated except for
certain edge cases.
This is useful for consolidating multiple databases into a single one,
e.g. for analytics.
3. Other fine-grained levels base on the object type such as index,
function, procedure and view etc.
Allows DDLs on certain object types to be replicated. At the moment
I’m unsure of a use case for this.
To implement such DDL replication levels, we need to modify the CREATE
PUBLICATION syntax. For example, to help starting the discussion on
the granularity of DDL replication, we can add a new option list ‘ddl’
in the WITH definition:
CREATE PUBLICATION mypub FOR ALL TABLES with
(publish = ‘insert, update, delete, truncate’, ddl = ‘database’)
CREATE PUBLICATION mypub FOR TABLE T1, T2 with
(publish = ‘insert, update, delete, truncate’, ddl = ‘table’)
CREATE PUBLICATION mypub FOR TABLE T1, T2 with
(publish = ‘insert, update, delete, truncate’, ddl = ‘table, function,
procedure’)
We can probably make “ddl = ‘database’” valid only for the FOR ALL
TABLES publication, because it doesn’t really make sense to replicate
all DDLs for a database when only a subset of tables are being
replicated (which can cause edge case 1 to occur frequently). “ddl =
‘database’” + FOR ALL TABLES is likely where logical replication of
DDL is most useful, i.e. for online major versions upgrades.
Based on the DDL publication levels we can further implement the logic
to conditionally log the DDL commands or to conditionally decode/ship
the logical DDL message.
Thoughts? Your feedback is appreciated.
Thanks,
Zheng Li
Amazon RDS/Aurora for PostgreSQL
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-02-22 10:05 Aleksander Alekseev <aleksander@timescale.com>
parent: Zheng Li <zhengli10@gmail.com>
2 siblings, 1 reply; 449+ messages in thread
From: Aleksander Alekseev @ 2022-02-22 10:05 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Zheng,
> I’m working on a patch to support logical replication of data
> definition language statements (DDLs).
That's great!
> However, there are still many edge cases to sort out because not every
> DDL statement can/should be replicated.
Maybe the first implementation shouldn't be perfect as long as known
limitations are documented and the future improvements are unlikely to
break anything for the users. Committing an MVP and iterating on this is
much simpler in terms of development and code review. Also, it will deliver
value to the users and give us feedback sooner.
> 1. DDL involving multiple tables where only some tables are replicated,
e.g.
>
> DROP TABLE replicated_foo, notreplicated_bar;
>
I would add DROP TABLE ... CASCADE to the list. Also, let's not forget that
PostgreSQL supports table inheritance and table partitioning.
> 2. Any DDL that calls a volatile function, such as NOW() or RAND(), is
> likely to generate a different value on each replica. It is possible
> to work around these issues—for example, the publisher can replace any
> volatile function calls with a fixed return value when the statement
> is logged so that the subscribers all get the same value. We will have
> to consider some other cases.
That would make sense.
> [...]
> Whether a DDL should be replicated also depends on what granularity do
> we define DDL replication. For example, we can define DDL replication
> on these levels:
>
> 1. Database level
> Allows all DDLs for a database to be replicated except for certain
> edge cases (refer to the edge cases mentioned above).
> This is likely a major use case, such as in online major version upgrade.
To my knowledge, this is not a primary use case for logical replication.
Also, I suspect that implementing it may be a bit challenging. What if we
focus on table-level replication for now?
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-02-22 12:00 Marcos Pegoraro <marcos@f10.com.br>
parent: Zheng Li <zhengli10@gmail.com>
2 siblings, 0 replies; 449+ messages in thread
From: Marcos Pegoraro @ 2022-02-22 12:00 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Em seg., 21 de fev. de 2022 às 13:13, Zheng Li <zhengli10@gmail.com>
escreveu:
>
> 2. Table level
> Allows DDLs on the published tables to be replicated except for
> certain edge cases.
>
> Think how to handle triggers and functions with same name but different
purpose.
Publisher
create function public.audit() returns trigger language plpgsql as $$
begin
new.Audit_User = current_user();
new.Audit_Date_Time = now();
return new;
end;$$
create trigger audit before insert or update on foo for each row execute
procedure public.audit();
Subscriber
create function public.audit() returns trigger language plpgsql as $$
begin
insert into Audit(Audit_Date_Time, Audit_User, Schema_Name, Table_Name,
Audit_Action, Field_Values) values(new.Audit_ts, new.Audit_User,
tg_table_schema, tg_table_name, tg_op, row_to_json(case when tg_op =
'DELETE' then old.* else new.* end));
return null;
end;$$
create trigger audit after insert or update or delete on foo for each row
execute procedure public.audit();
regards,
Marcos
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-02-24 03:10 Zheng Li <zhengli10@gmail.com>
parent: Aleksander Alekseev <aleksander@timescale.com>
0 siblings, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-02-24 03:10 UTC (permalink / raw)
To: Aleksander Alekseev <aleksander@timescale.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Aleksander,
>That's great!
Thanks!
>Maybe the first implementation shouldn't be perfect as long as known
>limitations are documented and the future improvements are unlikely to
>break anything for the users. Committing an MVP and iterating on this is
>much simpler in terms of development and code review. Also, it will deliver
>value to the users and give us feedback sooner.
Agree, I’ll post the prototype when it’s in a bit better shape. Also
I’d like to hear
some more feedback from the community on whether I’m heading the right
direction.
>I would add DROP TABLE ... CASCADE to the list. Also, let's not forget that
>PostgreSQL supports table inheritance and table partitioning.
Thanks, I will keep this in mind.
>> 1. Database level
>> Allows all DDLs for a database to be replicated except for certain
>> edge cases (refer to the edge cases mentioned above).
>> This is likely a major use case, such as in online major version upgrade.
>To my knowledge, this is not a primary use case for logical replication.
>Also, I suspect that implementing it may be a bit challenging. What if we
>focus on table-level replication for now?
I think it is due to the fact that the current limitations in logical
replication are
holding it back in major version upgrade (MVU). Online / reduced downtime MVU
is a popular request from customers, and why we should enhance logical
replication
to support this use case.
Also I think table-level DDL replication is actually more challenging,
especially in
the FOR TABLE case, due to the fact that differences are expected to
occur between the
source and target database. Marcos’ comment also justifies the complexity
in this case. Whereas database-level DDL replication in the FOR ALL
TABLE case is
relatively simple because the source and target database are (almost) identical.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-02-24 08:56 Aleksander Alekseev <aleksander@timescale.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Aleksander Alekseev @ 2022-02-24 08:56 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Zheng,
> >Also, I suspect that implementing it may be a bit challenging. What if we
> >focus on table-level replication for now?
>
> I think it is due to the fact that the current limitations in logical
> replication are
> holding it back in major version upgrade (MVU). Online / reduced downtime MVU
> is a popular request from customers, and why we should enhance logical
> replication
> to support this use case.
>
> Also I think table-level DDL replication is actually more challenging,
> especially in
> the FOR TABLE case, due to the fact that differences are expected to
> occur between the
> source and target database. Marcos’ comment also justifies the complexity
> in this case. Whereas database-level DDL replication in the FOR ALL
> TABLE case is
> relatively simple because the source and target database are (almost) identical.
Sure, I don't insist on implementing table-level replication first.
It's up to you. My point was that it's not necessary to implement
everything at once.
--
Best regards,
Aleksander Alekseev
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-13 11:35 Dilip Kumar <dilipbalaut@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
2 siblings, 2 replies; 449+ messages in thread
From: Dilip Kumar @ 2022-03-13 11:35 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Feb 21, 2022 at 9:43 PM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hello,
>
> One of the most frequently requested improvements from our customers
> is to reduce downtime associated with software updates (both major and
> minor versions). To do this, we have reviewed potential contributions to
> improving logical replication.
>
> I’m working on a patch to support logical replication of data
> definition language statements (DDLs). This is a useful feature when a
> database in logical replication has lots of tables, functions and
> other objects that change over time, such as in online cross major
> version upgrade.
+1
> I put together a prototype that replicates DDLs using the generic
> messages for logical decoding. The idea is to log the candidate DDL
> string in ProcessUtilitySlow() using LogLogicalMessge() with a new
> flag in WAL record type xl_logical_message indicating it’s a DDL
> message. The xl_logical_message record is decoded and sent to the
> subscriber via pgoutput. The logical replication worker process is
> dispatched for this new DDL message type and executes the command
> accordingly.
If you don't mind, would you like to share the POC or the branch for this work?
> However, there are still many edge cases to sort out because not every
> DDL statement can/should be replicated. Some of these include:
> 3. CREATE TABLE AS and SELECT INTO, For example:
>
> CREATE TABLE foo AS
> SELECT field_1, field_2 FROM bar;
>
> There are a few issues that can occur here. For one, it’s possible
> that table bar doesn't exist on the subscriber. Even if “bar” does
> exist, it may not be fully up-to-date with the publisher, which would
> cause a data mismatch on “foo” between the publisher and subscriber.
In such cases why don't we just log the table creation WAL for DDL
instead of a complete statement which creates the table and inserts
the tuple? Because we are already WAL logging individual inserts and
once you make sure of replicating the table creation I think the exact
data insertion on the subscriber side will be taken care of by the
insert WALs no?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-15 06:27 rajesh singarapu <rajesh.rs0541@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: rajesh singarapu @ 2022-03-15 06:27 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
+
On Sun, Mar 13, 2022 at 5:05 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Mon, Feb 21, 2022 at 9:43 PM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > Hello,
> >
> > One of the most frequently requested improvements from our customers
> > is to reduce downtime associated with software updates (both major and
> > minor versions). To do this, we have reviewed potential contributions to
> > improving logical replication.
> >
> > I’m working on a patch to support logical replication of data
> > definition language statements (DDLs). This is a useful feature when a
> > database in logical replication has lots of tables, functions and
> > other objects that change over time, such as in online cross major
> > version upgrade.
>
> +1
+1
> > I put together a prototype that replicates DDLs using the generic
> > messages for logical decoding. The idea is to log the candidate DDL
> > string in ProcessUtilitySlow() using LogLogicalMessge() with a new
> > flag in WAL record type xl_logical_message indicating it’s a DDL
> > message. The xl_logical_message record is decoded and sent to the
> > subscriber via pgoutput. The logical replication worker process is
> > dispatched for this new DDL message type and executes the command
> > accordingly.
>
> If you don't mind, would you like to share the POC or the branch for this work?
I would love to try this patch out, would you like to share branch or POC ?
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-16 21:17 Zheng Li <zhengli10@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
1 sibling, 4 replies; 449+ messages in thread
From: Zheng Li @ 2022-03-16 21:17 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
Hi,
>If you don't mind, would you like to share the POC or the branch for this work?
The POC patch is attached. It currently supports the following functionalities:
1. Configure either database level or table level DDL replication via
the CREATE PUBLICATION command.
2.Supports replication of DDL of the following types when database
level DDL replication is turned on. Other less common DDL types could
be added later.
T_CreateSchemaStmt
T_CreateStmt
T_CreateForeignTableStmt
T_AlterDomainStmt
T_DefineStmt
T_CompositeTypeStmt
T_CreateEnumStmt
T_CreateRangeStmt
T_AlterEnumStmt
T_ViewStmt
T_CreateFunctionStmt
T_AlterFunctionStmt
T_CreateTrigStmt
T_CreateDomainStmt
T_CreateCastStmt
T_CreateOpClassStmt
T_CreateOpFamilyStmt
T_AlterOpFamilyStmt
T_AlterOperatorStmt
T_AlterTypeStmt
T_GrantStmt
T_AlterCollationStmt
T_AlterTableStmt
T_IndexStmt
3.Supports replication of DDLs of the following types if only table
level DDL replication is turned on.
T_AlterTableStmt
T_IndexStmt
4.Supports seamless DML replication of new tables created via DDL
replication without having to manually running “ALTER SUBSCRIPTION ...
REFRESH PUBLICATION" on the subscriber.
Here is a demo:
source_db=# create publication mypub FOR ALL TABLES with (ddl = ‘database’);
target_db=# create subscription mysub CONNECTION 'dbname=source_db
host=localhost user=myuser port=5432' PUBLICATION mypub;
source_db=#
BEGIN;
CREATE TABLE foo (a int);
CREATE INDEX foo_idx ON foo (a);
ALTER TABLE foo ADD COLUMN b timestamptz;
CREATE FUNCTION foo_ts()
RETURNS trigger AS $$
BEGIN
NEW.b := current_timestamp;
RETURN NEW;
END;
$$
LANGUAGE plpgsql;
CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
FOR EACH ROW EXECUTE FUNCTION foo_ts();
INSERT INTO foo VALUES (1);
COMMIT;
source_db=# select * from foo;
a | b
---+-------------------------------
1 | 2022-03-15 19:07:53.005215+00
(1 row)
target_db=# select * from foo;
a | b
---+-------------------------------
1 | 2022-03-15 19:07:53.005215+00
(1 row)
Here is the remaining work for this patch:
1. Test and handle corner case DDLs, such as the ones listed in my
first email[1].
2. Investigate less common DDL types (categorized in function
LogLogicalDDLCommand) and decide if and under what conditions they
should be replicated.
3. Determine if we want to include table-level DDL replication. If so,
we need to continue to enhance and test this feature.
I think this will be ready to consider for commit once 1 and 2 are
resolved. I think 3 can be done in the second phase.
If you are reviewing the code, I’ve provided a detailed description of
the patch below:
1. Syntax, pg_publication and pg_dump change:
Allows the user to configure either database level or table level DDL
replication via the CREATE PUBLICATION command as proposed in my first
email. Two new columns are added to the pg_publication catalog to show
the DDL replication levels, test output publication.out is updated
accordingly. pg_dump is also modified to accommodate the
pg_publication catalog change.
2. Logical logging change
a. A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current consumers.
b. The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on the
configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
3. Logical decoding and Reorderbuffer change
Supports logical decoding of the new WAL record
xl_logical_ddl_message. This is similar to the logical decoding of
xl_logical_message. Tests for this change are added in the
test_decoding plugin.
4. Integration with pgoutput
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced for
this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
5. Logical replication worker change
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to SUBREL_STATE_INIT, So
that DML replication can progress on this new table without manually
running "ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
6. TAP test
A new TAP test 030_rep_ddl.pl is added. We mainly focused on testing
the happy path of database level replication so far. Corner case DDLs
and table level DDL replication are still to be carefully tested.
>> However, there are still many edge cases to sort out because not every
>> DDL statement can/should be replicated. Some of these include:
>> 3. CREATE TABLE AS and SELECT INTO, For example:
>>
>> CREATE TABLE foo AS
>> SELECT field_1, field_2 FROM bar;
>>
>> There are a few issues that can occur here. For one, it’s possible
>> that table bar doesn't exist on the subscriber. Even if “bar” does
>> exist, it may not be fully up-to-date with the publisher, which would
>> cause a data mismatch on “foo” between the publisher and subscriber.
>In such cases why don't we just log the table creation WAL for DDL
>instead of a complete statement which creates the table and inserts
>the tuple? Because we are already WAL logging individual inserts and
>once you make sure of replicating the table creation I think the exact
>data insertion on the subscriber side will be taken care of by the
>insert WALs no?
The table creation WAL and table insert WAL are available. The tricky
part is how do we break down this command into two parts (a normal
CREATE TABLE followed by insertions) either from the parsetree or the
WALs. I’ll have to dig more on this.
I look forward to your feedback.
Regards,
Zheng
[1] https://www.postgresql.org/message-id/CAAD30U+pVmfKwUKy8cbZOnUXyguJ-uBNejwD75Kyo=OjdQGJ9g@mail.gmail...
On Sun, Mar 13, 2022 at 7:35 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Mon, Feb 21, 2022 at 9:43 PM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > Hello,
> >
> > One of the most frequently requested improvements from our customers
> > is to reduce downtime associated with software updates (both major and
> > minor versions). To do this, we have reviewed potential contributions to
> > improving logical replication.
> >
> > I’m working on a patch to support logical replication of data
> > definition language statements (DDLs). This is a useful feature when a
> > database in logical replication has lots of tables, functions and
> > other objects that change over time, such as in online cross major
> > version upgrade.
>
> +1
>
> > I put together a prototype that replicates DDLs using the generic
> > messages for logical decoding. The idea is to log the candidate DDL
> > string in ProcessUtilitySlow() using LogLogicalMessge() with a new
> > flag in WAL record type xl_logical_message indicating it’s a DDL
> > message. The xl_logical_message record is decoded and sent to the
> > subscriber via pgoutput. The logical replication worker process is
> > dispatched for this new DDL message type and executes the command
> > accordingly.
>
> If you don't mind, would you like to share the POC or the branch for this work?
>
> > However, there are still many edge cases to sort out because not every
> > DDL statement can/should be replicated. Some of these include:
>
> > 3. CREATE TABLE AS and SELECT INTO, For example:
> >
> > CREATE TABLE foo AS
> > SELECT field_1, field_2 FROM bar;
> >
> > There are a few issues that can occur here. For one, it’s possible
> > that table bar doesn't exist on the subscriber. Even if “bar” does
> > exist, it may not be fully up-to-date with the publisher, which would
> > cause a data mismatch on “foo” between the publisher and subscriber.
>
> In such cases why don't we just log the table creation WAL for DDL
> instead of a complete statement which creates the table and inserts
> the tuple? Because we are already WAL logging individual inserts and
> once you make sure of replicating the table creation I think the exact
> data insertion on the subscriber side will be taken care of by the
> insert WALs no?
>
>
> --
> Regards,
> Dilip Kumar
> EnterpriseDB: http://www.enterprisedb.com
Attachments:
[application/octet-stream] ddl_replication.patch1 (149.6K, ../../CAAD30ULtoGp8L_GKbV15Wnm+X5r=SE7MOnYHuqBr396m26jJSA@mail.gmail.com/2-ddl_replication.patch1)
download
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-16 22:06 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Zheng Li <zhengli10@gmail.com>
3 siblings, 2 replies; 449+ messages in thread
From: Alvaro Herrera @ 2022-03-16 22:06 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
Hello
I think this is a pretty interesting and useful feature.
Did you see some old code I wrote towards this goal?
https://www.postgresql.org/message-id/20150215044814.GL3391@alvh.no-ip.org
The intention was that DDL would produce some JSON blob that accurately
describes the DDL that was run; the caller can acquire that and use it
to produce working DDL that doesn't depend on runtime conditions. There
was lots of discussion on doing things this way. It was ultimately
abandoned, but I think it's valuable.
--
Álvaro Herrera Valdivia, Chile — https://www.EnterpriseDB.com/
"Porque francamente, si para saber manejarse a uno mismo hubiera que
rendir examen... ¿Quién es el machito que tendría carnet?" (Mafalda)
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-17 00:08 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
3 siblings, 0 replies; 449+ messages in thread
From: Japin Li @ 2022-03-17 00:08 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com; pgsql-hackers@lists.postgresql.org
Hi, Zhang Li
On Thu, 17 Mar 2022 at 05:17, Zheng Li <zhengli10@gmail.com> wrote:
> Hi,
>
>>If you don't mind, would you like to share the POC or the branch for this work?
>
> The POC patch is attached. It currently supports the following functionalities:
> 1. Configure either database level or table level DDL replication via
> the CREATE PUBLICATION command.
>
> 2.Supports replication of DDL of the following types when database
> level DDL replication is turned on. Other less common DDL types could
> be added later.
> T_CreateSchemaStmt
> T_CreateStmt
> T_CreateForeignTableStmt
> T_AlterDomainStmt
> T_DefineStmt
> T_CompositeTypeStmt
> T_CreateEnumStmt
> T_CreateRangeStmt
> T_AlterEnumStmt
> T_ViewStmt
> T_CreateFunctionStmt
> T_AlterFunctionStmt
> T_CreateTrigStmt
> T_CreateDomainStmt
> T_CreateCastStmt
> T_CreateOpClassStmt
> T_CreateOpFamilyStmt
> T_AlterOpFamilyStmt
> T_AlterOperatorStmt
> T_AlterTypeStmt
> T_GrantStmt
> T_AlterCollationStmt
> T_AlterTableStmt
> T_IndexStmt
>
> 3.Supports replication of DDLs of the following types if only table
> level DDL replication is turned on.
> T_AlterTableStmt
> T_IndexStmt
>
> 4.Supports seamless DML replication of new tables created via DDL
> replication without having to manually running “ALTER SUBSCRIPTION ...
> REFRESH PUBLICATION" on the subscriber.
>
Great! I think we can split the patch into several pieces (at least
implementation and test cases) for easier review.
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
There is a typo, s/Simiarl/Similar/.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-17 15:56 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
3 siblings, 0 replies; 449+ messages in thread
From: Japin Li @ 2022-03-17 15:56 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com; pgsql-hackers@lists.postgresql.org
On Thu, 17 Mar 2022 at 05:17, Zheng Li <zhengli10@gmail.com> wrote:
> Hi,
>
>>If you don't mind, would you like to share the POC or the branch for this work?
>
> The POC patch is attached. It currently supports the following functionalities:
Hi,
When I try to run regression test, there has some errors.
make[2]: Entering directory '/home/px/Codes/postgresql/Debug/src/test/subscription'
rm -rf '/home/px/Codes/postgresql/Debug/src/test/subscription'/tmp_check && /usr/bin/mkdir -p '/home/px/Codes/postgresql/Debug/src/test/subscription'/tmp_check && cd /home/px/Codes/postgresql/Debug/../src
/test/subscription && TESTDIR='/home/px/Codes/postgresql/Debug/src/test/subscription' PATH="/home/px/Codes/postgresql/Debug/tmp_install/home/px/Codes/postgresql/Debug/pg/bin:/home/px/Codes/postgresql/Debu
g/src/test/subscription:$PATH" LD_LIBRARY_PATH="/home/px/Codes/postgresql/Debug/tmp_install/home/px/Codes/postgresql/Debug/pg/lib:$LD_LIBRARY_PATH" PGPORT='65432' PG_REGRESS='/home/px/Codes/postgresql/De
bug/src/test/subscription/../../../src/test/regress/pg_regress' /usr/bin/prove -I /home/px/Codes/postgresql/Debug/../src/test/perl/ -I /home/px/Codes/postgresql/Debug/../src/test/subscription t/*.pl
t/001_rep_changes.pl ............... ok
t/002_types.pl ..................... ok
t/003_constraints.pl ............... ok
t/004_sync.pl ...................... 6/? # Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 29 just after 7.
t/004_sync.pl ...................... Dubious, test returned 29 (wstat 7424, 0x1d00)
All 7 subtests passed
t/005_encoding.pl .................. ok
t/006_rewrite.pl ................... 1/? # poll_query_until timed out executing this query:
# SELECT '0/14A8648' <= replay_lsn AND state = 'streaming' FROM pg_catalog.pg_stat_replication WHERE application_name = 'mysub';
# expecting this output:
# t
# last actual query output:
#
# with stderr:
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 29 just after 1.
t/006_rewrite.pl ................... Dubious, test returned 29 (wstat 7424, 0x1d00)
All 1 subtests passed
t/007_ddl.pl ....................... ok
t/008_diff_schema.pl ............... 1/? # Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 29 just after 4.
t/008_diff_schema.pl ............... Dubious, test returned 29 (wstat 7424, 0x1d00)
All 4 subtests passed
t/009_matviews.pl .................. Dubious, test returned 29 (wstat 7424, 0x1d00)
No subtests run
The new patch change the behavior of publication, when we drop a table
on publisher, the table also be dropped on subscriber, and this made the
regression testa failed, since the regression test will try to drop the
table on subscriber.
IMO, we can disable the DDLs replication by default, which makes the
regression test happy. Any thoughts?
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-17 16:22 Zheng Li <zhengli10@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-03-17 16:22 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; Japin Li <japinli@hotmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
Hello Alvaro,
> I think this is a pretty interesting and useful feature.
>
> Did you see some old code I wrote towards this goal?
> https://www.postgresql.org/message-id/20150215044814.GL3391@alvh.no-ip.org
> The intention was that DDL would produce some JSON blob that accurately
> describes the DDL that was run; the caller can acquire that and use it
> to produce working DDL that doesn't depend on runtime conditions. There
> was lots of discussion on doing things this way. It was ultimately
> abandoned, but I think it's valuable.
Thanks for pointing this out. I'm looking into it.
Hello Japin,
>The new patch change the behavior of publication, when we drop a table
>on publisher, the table also be dropped on subscriber, and this made the
>regression testa failed, since the regression test will try to drop the
>table on subscriber.
>IMO, we can disable the DDLs replication by default, which makes the
>regression test happy. Any thoughts?
Good catch, I forgot to run the whole TAP test suite. I think for now
we can simply disable DDL replication for the failing tests when
creating publication: $node_publisher->safe_psql('postgres',
"CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl='')");
I'll fix it in the next version of patch. I'll also work on breaking
down the patch into smaller pieces for ease of review.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 00:18 Zheng Li <zhengli10@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-03-18 00:18 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hello,
Attached please find the broken down patch set. Also fixed the failing
TAP tests Japin reported.
Regards,
Zheng Li
Amazon RDS/Aurora for PostgreSQL
Attachments:
[application/octet-stream] 0001-syntax-pg_publication-pg_dump-ddl_replication.patch (62.6K, ../../CAAD30UKTp87+kvGZYL3M2Suxq=WEvFUG24ZRT0yT9rqdkP=uMA@mail.gmail.com/2-0001-syntax-pg_publication-pg_dump-ddl_replication.patch)
download | inline diff:
commit 3caa9210974f749e7bf82d80de57fe9d208984b4
Author: Zheng (Zane) Li <zhelli@amazon.com>
Date: Fri Mar 18 00:01:34 2022 +0000
0001-syntax-pg_publication-pg_dump-ddl_replication.patch
1. Syntax, pg_publication and pg_dump change:
Allows the user to configure either database level or table level
DDL replication via the CREATE PUBLICATION command as proposed
in my first email. Two new columns are added to the pg_publication
catalog to show the DDL replication levels, test output
publication.out is updated accordingly. pg_dump is also modified
to accommodate the pg_publication catalog change.
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 789b895db8..00b5673b8f 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -857,6 +857,8 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubactions.pubddl_database = pubform->pubddl_database;
+ pub->pubactions.pubddl_table = pubform->pubddl_table;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 1aad2e769c..7d767c2bc4 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -80,10 +80,12 @@ static void PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok);
static void
parse_publication_options(ParseState *pstate,
List *options,
+ bool for_all_tables,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *ddl_level_given)
{
ListCell *lc;
@@ -96,6 +98,16 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ if (for_all_tables)
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ else
+ {
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+ }
/* Parse options */
foreach(lc, options)
@@ -154,6 +166,57 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "ddl") == 0)
+ {
+ char *ddl_level;
+ List *ddl_list;
+ ListCell *lc;
+
+ if (*ddl_level_given)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options")));
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
+ ddl_level = defGetString(defel);
+
+ if (!SplitIdentifierString(ddl_level, ',', &ddl_list))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax for \"ddl\" option")));
+
+ /* Process the option list. */
+ foreach(lc, ddl_list)
+ {
+ char *publish_opt = (char *) lfirst(lc);
+
+ if (strcmp(publish_opt, "database") == 0)
+ {
+ if (!for_all_tables)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("DDL replication on the database level is only supported in FOR ALL TABLES publication")));
+ else
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ }
+ else if (strcmp(publish_opt, "table") == 0)
+ pubactions->pubddl_table = true;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized \"ddl\" value: \"%s\"", publish_opt)));
+ }
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -622,6 +685,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Datum values[Natts_pg_publication];
HeapTuple tup;
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -664,9 +728,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
parse_publication_options(pstate,
stmt->options,
+ stmt->for_all_tables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -683,6 +749,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubddl_database - 1] =
+ BoolGetDatum(pubactions.pubddl_database);
+ values[Anum_pg_publication_pubddl_table - 1] =
+ BoolGetDatum(pubactions.pubddl_table);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -766,6 +836,7 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
bool replaces[Natts_pg_publication];
Datum values[Natts_pg_publication];
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -774,11 +845,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
List *root_relids = NIL;
ListCell *lc;
+ pubform = (Form_pg_publication) GETSTRUCT(tup);
+
parse_publication_options(pstate,
stmt->options,
+ pubform->puballtables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -859,6 +934,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
+ if (ddl_level_given)
+ {
+ values[Anum_pg_publication_pubddl_database - 1] = BoolGetDatum(pubactions.pubddl_database);
+ replaces[Anum_pg_publication_pubddl_database - 1] = true;
+
+ values[Anum_pg_publication_pubddl_table - 1] = BoolGetDatum(pubactions.pubddl_table);
+ replaces[Anum_pg_publication_pubddl_table - 1] = true;
+ }
+
if (publish_via_partition_root_given)
{
values[Anum_pg_publication_pubviaroot - 1] = BoolGetDatum(publish_via_partition_root);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index fccffce572..d88b2aa642 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5606,6 +5606,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl_database |= pubform->pubddl_database;
+ pubdesc->pubactions.pubddl_table |= pubform->pubddl_table;
/*
* Check if all columns referenced in the filter expression are part of
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 725cd2e4eb..5e06cf4d3a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3819,6 +3819,8 @@ getPublications(Archive *fout, int *numPublications)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubddl_database;
+ int i_pubddl_table;
int i,
ntups;
@@ -3833,23 +3835,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubddl_database, p.pubddl_table "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3866,6 +3874,8 @@ getPublications(Archive *fout, int *numPublications)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubddl_database = PQfnumber(res, "pubddl_database");
+ i_pubddl_table = PQfnumber(res, "pubddl_table");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3890,6 +3900,10 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubddl_database =
+ (strcmp(PQgetvalue(res, i, i_pubddl_database), "t") == 0);
+ pubinfo[i].pubddl_table =
+ (strcmp(PQgetvalue(res, i, i_pubddl_table), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -3969,6 +3983,29 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
appendPQExpBufferStr(query, "'");
+ appendPQExpBufferStr(query, ", ddl = '");
+ first = true;
+
+ if (pubinfo->pubddl_database)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "database");
+ first = false;
+ }
+
+ if (pubinfo->pubddl_table)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "table");
+ first = false;
+ }
+
+ appendPQExpBufferStr(query, "'");
+
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 772dc0cf7a..17789ea271 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,8 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 991bfc1546..7a7d4a50e2 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -5820,7 +5820,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -5855,6 +5855,15 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_database AS \"%s\"",
+ gettext_noop("Database level DDLs"));
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_table AS \"%s\"",
+ gettext_noop("Table level DDLs"));
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -5936,6 +5945,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -5952,6 +5962,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -5965,6 +5976,11 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ {
+ appendPQExpBufferStr(&buf,
+ ", pubddl_database, pubddl_table");
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6011,6 +6027,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols += 2;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6025,6 +6043,11 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ {
+ printTableAddHeader(&cont, gettext_noop("Database level DDL"), true, align);
+ printTableAddHeader(&cont, gettext_noop("Table level DDL"), true, align);
+ }
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6035,6 +6058,11 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ {
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
+ printTableAddCell(&cont, PQgetvalue(res, i, 10), false, false);
+ }
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index fe773cf9b7..b86b49aafc 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -20,6 +20,7 @@
#include "catalog/genbki.h"
#include "catalog/objectaddress.h"
#include "catalog/pg_publication_d.h"
+#include "tcop/utility.h"
/* ----------------
* pg_publication definition. cpp turns this into
@@ -54,6 +55,12 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if database level ddls are published */
+ bool pubddl_database;
+
+ /* true if table level ddls are published */
+ bool pubddl_table;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +79,8 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationActions;
typedef struct PublicationDesc
@@ -146,5 +155,6 @@ extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..073f46ef1a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -15,6 +15,7 @@
#define DEFREM_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_publication.h"
#include "nodes/params.h"
#include "parser/parse_node.h"
#include "tcop/dest.h"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 4e191c120a..6761e3bbc0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | t | t | t | f | f | f | t | t
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -658,10 +658,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -699,10 +699,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -780,10 +780,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -793,20 +793,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f | f
(1 row)
-- adding schemas and tables
@@ -822,19 +822,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -848,44 +848,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -919,10 +919,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -930,20 +930,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -951,10 +951,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -963,10 +963,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -975,10 +975,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -986,10 +986,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -997,10 +997,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1008,29 +1008,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1039,10 +1039,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1051,10 +1051,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1124,18 +1124,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1145,20 +1145,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
[application/octet-stream] 0002-logical_decoding_ddl_message.patch (57.5K, ../../CAAD30UKTp87+kvGZYL3M2Suxq=WEvFUG24ZRT0yT9rqdkP=uMA@mail.gmail.com/3-0002-logical_decoding_ddl_message.patch)
download | inline diff:
commit b2c35793dd28b9c5c3c06f0d81d1981009b478e1
Author: Zheng (Zane) Li <zhelli@amazon.com>
Date: Fri Mar 18 00:04:35 2022 +0000
0002-logical_decoding_ddl_message.patch
2. Logical logging change
a. A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current
consumers.
b. The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on
the configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
3. Logical decoding and Reorderbuffer change
Supports logical decoding of the new WAL record
xl_logical_ddl_message. This is similar to the logical decoding of
xl_logical_message. Tests for this change are added in the
test_decoding plugin.
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index 36929dd97d..d5091c568d 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -4,7 +4,7 @@ MODULES = test_decoding
PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
- decoding_into_rel binary prepared replorigin time messages \
+ decoding_into_rel binary prepared replorigin time messages ddlmessages\
spill slot truncate stream stats twophase twophase_stream \
sequence
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
new file mode 100644
index 0000000000..a92a959cad
--- /dev/null
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -0,0 +1,48 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+ALTER TABLE tab1 drop c3;
+DROP TABLE tab1;
+BEGIN;
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+ROLLBACK;
+BEGIN;
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+COMMIT;
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE tab1;
+DROP publication mypub;
+ data
+---------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 47 content:CREATE TABLE tab1 (id serial unique, data int);
+ BEGIN
+ sequence public.tab1_id_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
+ COMMIT
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 32 content:ALTER TABLE tab1 add c3 varchar;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 25 content:ALTER TABLE tab1 drop c3;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 16 content:DROP TABLE tab1;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 47 content:CREATE TABLE tab1 (id serial unique, data int);
+ BEGIN
+ sequence public.tab1_id_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 32 content:ALTER TABLE tab1 add c3 varchar;
+ COMMIT
+(12 rows)
+
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
new file mode 100644
index 0000000000..e4fc3d89bd
--- /dev/null
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -0,0 +1,28 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+ALTER TABLE tab1 drop c3;
+DROP TABLE tab1;
+
+BEGIN;
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+ROLLBACK;
+
+BEGIN;
+CREATE TABLE tab1 (id serial unique, data int);
+ALTER TABLE tab1 add c3 varchar;
+COMMIT;
+
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE tab1;
+DROP publication mypub;
+
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index ea22649e41..5e02b274c5 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -77,6 +77,11 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_sequence(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr sequence_lsn,
Relation rel, bool transactional,
@@ -121,6 +126,11 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_stream_sequence(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr sequence_lsn,
Relation rel, bool transactional,
@@ -150,6 +160,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
cb->message_cb = pg_decode_message;
+ cb->ddlmessage_cb = pg_decode_ddlmessage;
cb->sequence_cb = pg_decode_sequence;
cb->filter_prepare_cb = pg_decode_filter_prepare;
cb->begin_prepare_cb = pg_decode_begin_prepare_txn;
@@ -163,6 +174,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pg_decode_stream_commit;
cb->stream_change_cb = pg_decode_stream_change;
cb->stream_message_cb = pg_decode_stream_message;
+ cb->stream_ddlmessage_cb = pg_decode_stream_ddlmessage;
cb->stream_sequence_cb = pg_decode_stream_sequence;
cb->stream_truncate_cb = pg_decode_stream_truncate;
}
@@ -758,7 +770,8 @@ pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
appendStringInfo(ctx->out, "message: transactional: %d prefix: %s, sz: %zu content:",
@@ -799,6 +812,19 @@ pg_decode_sequence(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+ appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_stream_start(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
@@ -979,7 +1005,8 @@ pg_decode_stream_change(LogicalDecodingContext *ctx,
static void
pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
@@ -991,7 +1018,35 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
else
{
appendStringInfo(ctx->out, "streaming message: transactional: %d prefix: %s, sz: %zu content:",
- transactional, prefix, sz);
+ transactional, prefix, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ }
+
+ OutputPluginWrite(ctx, true);
+}
+
+/*
+ * In streaming mode, we don't display the contents for transactional messages
+ * as the transaction can abort at a later point in time. We don't want users to
+ * see the message contents until the transaction is committed.
+ */
+static void
+pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char * role, const char * search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+
+ if (transactional)
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
+ transactional, prefix, role, search_path, sz);
+ }
+ else
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
}
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72fd86..b8e29e8df3 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000000..7a352d540a
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *role = xlrec->message + xlrec->prefix_size;
+ char *search_path = xlrec->message + xlrec->prefix_size + xlrec->role_size;
+ char *message = xlrec->message + xlrec->prefix_size + xlrec->role_size + xlrec->search_path_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
+ xlrec->transactional ? "transactional" : "non-transactional",
+ prefix, role, search_path, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index f8847d5aeb..b761662d28 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -26,6 +26,7 @@
#include "commands/tablespace.h"
#include "replication/decode.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "storage/standby.h"
#include "utils/relmapper.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 00b5673b8f..f2d11c3847 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1007,3 +1007,55 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
+
+/*
+ * Checks if DDL on relation (relid) need xlog for logical replication
+ */
+bool
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+{
+ List *allTablePubs = NIL;
+ List *tablePubs = NIL;
+ ListCell *lc;
+
+ /* Only replicate toplevel DDL command */
+ if (!isTopLevel)
+ return false;
+ if (relid == InvalidOid && !forAllTabPubOnly)
+ return false;
+
+ /*
+ * Log the DDL command if
+ * there is any FOR ALL TABLES publication with pubddl_database on
+ * or
+ * this TABLE belongs to any non FOR ALL publications with pubddl_table on
+ */
+ allTablePubs = GetAllTablesPublications();
+ foreach(lc, allTablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_database)
+ return true;
+ }
+
+ /*
+ * If forAllTabPubOnly is true (i.e. database level replication is required for the DDL
+ * to be logged), we can bail now since no publication has been found with pubddl_database on
+ */
+ if (forAllTabPubOnly)
+ return false;
+
+ tablePubs = GetRelationPublications(relid);
+ foreach(lc, tablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_table)
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc5872f988..f56ea4c49c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -79,6 +79,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -1325,13 +1326,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(DropStmt *drop)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
+ bool ddlxlog = XLogLogicalInfoActive();
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1427,10 +1429,37 @@ RemoveRelations(DropStmt *drop)
/* Not there? */
if (!OidIsValid(relOid))
{
+ ddlxlog = false;
DropErrorMsgNonExistent(rel, relkind, drop->missing_ok);
continue;
}
+ /*
+ * Only log DROP RELATION cmd for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on or
+ * every relation to be dropped belongs to any non FOR ALL publications with pubddl_table on
+ */
+ if (ddlxlog)
+ {
+ Oid tableOid = InvalidOid;
+
+ if (relkind == RELKIND_RELATION)
+ tableOid = relOid;
+ else if (relkind == RELKIND_INDEX)
+ tableOid = IndexGetRelation(relOid, true);
+ /*
+ * Other relation types require database level ddl replication and are
+ * already logged in LogLogicalDDLCommand() if needed.
+ */
+ else
+ ddlxlog = false;
+
+ /* DROP RELATION or INDEX are allowed in table level DDL replication */
+ if (tableOid != InvalidOid &&
+ !ddl_need_xlog(tableOid, false, isTopLevel))
+ ddlxlog = false;
+ }
+
/*
* Decide if concurrent mode needs to be used here or not. The
* relation persistence cannot be known without its OID.
@@ -1462,6 +1491,18 @@ RemoveRelations(DropStmt *drop)
add_exact_object_address(&obj, objects);
}
+ /* Log the Drop command for logical replication */
+ if (ddlxlog)
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
performMultipleDeletions(objects, drop->behavior, flags);
free_object_addresses(objects);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fdeb71..f3eeb67312 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000000..f93573079a
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
+ * Transactional messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ * Non-transactional messages are sent to the plugin at the time when the
+ * logical decoding reads them from XLOG. This also means that transactional
+ * messages won't be delivered if the transaction was rolled back but the
+ * non-transactional one will always be delivered.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
+{
+ xl_logical_ddl_message xlrec;
+ const char *role;
+
+ role = GetUserNameFromId(roleoid, false);
+
+ /*
+ * Force xid to be allocated if we're emitting a transactional message.
+ */
+ if (transactional)
+ {
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+ }
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.transactional = transactional;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.role_size = strlen(role) + 1;
+ xlrec.search_path_size = strlen(namespace_search_path) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, role), xlrec.role_size);
+ XLogRegisterData(namespace_search_path, xlrec.search_path_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 8c00a73cb9..b9f3086e2c 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -605,6 +606,61 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
message->message + message->prefix_size);
}
+/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ Snapshot snapshot;
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (message->transactional &&
+ !SnapBuildProcessChange(builder, xid, buf->origptr))
+ return;
+ else if (!message->transactional &&
+ (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
+ SnapBuildXactNeedsSkip(builder, buf->origptr)))
+ return;
+
+ snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ message->transactional,
+ message->message,
+ /* first part of message is prefix */
+ message->message + message->prefix_size,
+ /* Second part of message is role*/
+ message->message + message->prefix_size + message->role_size,
+ /* Third part of message is search_path */
+ message->message_size,
+ message->message + message->prefix_size +
+ message->role_size + message->search_path_size);
+}
+
/*
* Consolidated commit record handling between the different form of commit
* records.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 934aa13f2d..4e84705007 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void sequence_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr sequence_lsn, Relation rel,
bool transactional,
@@ -94,6 +98,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void stream_sequence_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr sequence_lsn, Relation rel,
bool transactional,
@@ -226,6 +234,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
ctx->reorder->sequence = sequence_cb_wrapper;
/*
@@ -243,6 +252,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_sequence_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
@@ -261,6 +271,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_sequence = stream_sequence_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1250,6 +1261,44 @@ sequence_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char *search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
@@ -1596,6 +1645,48 @@ stream_sequence_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char* search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c2d9be81fa..599dcef6bf 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -555,6 +555,20 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.role != NULL)
+ pfree(change->data.ddlmsg.role);
+ change->data.ddlmsg.role = NULL;
+ if (change->data.ddlmsg.search_path != NULL)
+ pfree(change->data.ddlmsg.search_path);
+ change->data.ddlmsg.search_path = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -1140,6 +1154,64 @@ ReorderBufferQueueSequence(ReorderBuffer *rb, TransactionId xid,
}
}
+/*
+ * A transactional DDL message is queued to be processed upon commit and a
+ * non-transactional DDL message gets processed immediately.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
+{
+ if (transactional)
+ {
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ else
+ {
+ ReorderBufferTXN *txn = NULL;
+ volatile Snapshot snapshot_now = snapshot;
+
+ if (xid != InvalidTransactionId)
+ txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+
+ /* setup snapshot to allow catalog access */
+ SetupHistoricSnapshot(snapshot_now, NULL);
+ PG_TRY();
+ {
+ rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+
+ TeardownHistoricSnapshot(false);
+ }
+ PG_CATCH();
+ {
+ TeardownHistoricSnapshot(true);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+}
+
/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
@@ -2234,6 +2306,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message);
}
+/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
/*
* Helper function for ReorderBufferProcessTXN for applying sequences.
*/
@@ -2635,6 +2730,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -4034,6 +4133,53 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the role including the size */
+ memcpy(data, &role_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.role,
+ role_size);
+ data += role_size;
+
+ /* write the search_path including the size */
+ memcpy(data, &search_path_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.search_path,
+ search_path_size);
+ data += search_path_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4381,6 +4527,18 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4657,8 +4815,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4672,6 +4829,49 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+ Size role_size;
+ Size search_path_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read role */
+ memcpy(&role_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.role = MemoryContextAlloc(rb->context,
+ role_size);
+ memcpy(change->data.ddlmsg.role, data, role_size);
+ Assert(change->data.ddlmsg.role[role_size - 1] == '\0');
+ data += role_size;
+
+ /* read search_path */
+ memcpy(&search_path_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.search_path = MemoryContextAlloc(rb->context,
+ search_path_size);
+ memcpy(change->data.ddlmsg.search_path, data, search_path_size);
+ Assert(change->data.ddlmsg.search_path[search_path_size - 1] == '\0');
+ data += search_path_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3780c6e812..e5ec5713fa 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -62,6 +62,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
#include "storage/fd.h"
@@ -86,7 +87,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -986,7 +987,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel);
}
break;
@@ -1086,6 +1087,154 @@ standard_ProcessUtility(PlannedStmt *pstmt,
CommandCounterIncrement();
}
+/*
+ * Log a DDL command for logical replication
+ * Some DDLs are only replicated in Database Level DDL replication
+ * Some can be replicated in Table Level DDL replication.
+ *
+ * Currently we focus on supporting Database Level DDL replication
+ */
+static void
+LogLogicalDDLCommand(Node *parsetree, const char *queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* Fisrtly, commands that are only supported in Database level DDL replication */
+ case T_CreateSchemaStmt:
+ case T_CreateStmt:
+ case T_CreateForeignTableStmt:
+ case T_AlterDomainStmt:
+ case T_DefineStmt:
+ case T_CompositeTypeStmt:
+ case T_CreateEnumStmt:
+ case T_CreateRangeStmt:
+ case T_AlterEnumStmt:
+ case T_ViewStmt:
+ case T_CreateFunctionStmt:
+ case T_AlterFunctionStmt:
+ case T_CreateTrigStmt:
+ case T_CreateDomainStmt:
+ case T_CreateCastStmt:
+ case T_CreateOpClassStmt:
+ case T_CreateOpFamilyStmt:
+ case T_AlterOpFamilyStmt:
+ case T_AlterOperatorStmt:
+ case T_AlterTypeStmt:
+ case T_GrantStmt:
+ case T_AlterCollationStmt:
+ /*
+ * Log these stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+
+ /*
+ * Secondly, commands that may be allowed in Table level DDL replication.
+ * These are currently handled in the later execution path of the command.
+ * Because we need to get the relation id which readily available in later
+ * code path.
+ */
+ case T_AlterTableStmt:
+ case T_IndexStmt:
+ case T_RenameStmt: /* TODO */
+ case T_AlterOwnerStmt: /* TODO */
+ break;
+
+ /* DropStmt depends on the removeType */
+ case T_DropStmt:
+ {
+ DropStmt* stmt = (DropStmt*) parsetree;
+ switch (stmt->removeType)
+ {
+ /* Maybe allowed in Table level DDL replication, handled in later code path */
+ case OBJECT_INDEX:
+ case OBJECT_TABLE:
+ break;
+ /* Drop of sequence is by logical replication of sequences separately */
+ case OBJECT_SEQUENCE:
+ break;
+ /* Drop of other objects are allowed in Database level DDL replication only */
+ case OBJECT_VIEW:
+ case OBJECT_MATVIEW:
+ case OBJECT_FOREIGN_TABLE:
+ default:
+ /*
+ * Log these DropStmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+ }
+ }
+ /*
+ * Lastly, rule out DDLs we don't replicate yet in DDL replication
+ * Some of these can be supported, we just need to investigate and run tests.
+ */
+ case T_CreateExtensionStmt:
+ case T_AlterExtensionStmt:
+ case T_AlterExtensionContentsStmt:
+ case T_CreateFdwStmt:
+ case T_AlterFdwStmt:
+ case T_CreateForeignServerStmt:
+ case T_AlterForeignServerStmt:
+ case T_CreateUserMappingStmt:
+ case T_AlterUserMappingStmt:
+ case T_DropUserMappingStmt:
+ case T_ImportForeignSchemaStmt:
+ case T_RuleStmt:
+ case T_CreateSeqStmt:
+ case T_AlterSeqStmt:
+ case T_CreateTableAsStmt:
+ case T_RefreshMatViewStmt:
+ case T_CreatePLangStmt:
+ case T_CreateConversionStmt:
+ case T_CreateTransformStmt:
+ case T_AlterTSDictionaryStmt:
+ case T_AlterTSConfigurationStmt:
+ case T_AlterTableMoveAllStmt:
+ case T_AlterObjectDependsStmt:
+ case T_AlterObjectSchemaStmt:
+ case T_CommentStmt:
+ case T_DropOwnedStmt:
+ case T_AlterDefaultPrivilegesStmt:
+ case T_CreatePolicyStmt:
+ case T_AlterPolicyStmt:
+ case T_SecLabelStmt:
+ case T_CreateAmStmt:
+ case T_CreatePublicationStmt:
+ case T_AlterPublicationStmt:
+ case T_CreateSubscriptionStmt:
+ case T_AlterSubscriptionStmt:
+ case T_DropSubscriptionStmt:
+ case T_CreateStatsStmt:
+ case T_AlterStatsStmt:
+ break;
+ default:
+ break;
+ }
+}
+
/*
* The "Slow" variant of ProcessUtility should only receive statements
* supported by the event triggers facility. Therefore, we always
@@ -1118,6 +1267,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
switch (nodeTag(parsetree))
{
/*
@@ -1320,6 +1476,23 @@ ProcessUtilitySlow(ParseState *pstate,
EventTriggerAlterTableStart(parsetree);
EventTriggerAlterTableRelid(relid);
+ /*
+ * Log the ALTER TABLE command if
+ * There is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
/* ... and do it */
AlterTable(atstmt, lockmode, &atcontext);
@@ -1538,6 +1711,24 @@ ProcessUtilitySlow(ParseState *pstate,
/* ... and do it */
EventTriggerAlterTableStart(parsetree);
+
+ /*
+ * Log CREATE INDEX cmd for logical replication if
+ * there is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on.
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
address =
DefineIndex(relid, /* OID of heap relation */
stmt,
@@ -1754,7 +1945,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt((DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -1975,7 +2166,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
{
switch (stmt->removeType)
{
@@ -1990,7 +2181,7 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(stmt);
+ RemoveRelations(pstate, stmt, isTopLevel);
break;
default:
RemoveObjects(stmt);
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6a4ebd1310..c38c163f28 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index cf8b6d4819..5ffdc16cca 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 5d4037f26e..68781365de 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -18,6 +18,7 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "storage/lock.h"
#include "utils/relcache.h"
@@ -27,7 +28,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(DropStmt *drop);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000000..1e8ef22296
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ bool transactional; /* is message transactional? */
+ Size prefix_size; /* length of prefix */
+ Size role_size; /* length of the role that executes the DDL command */
+ Size search_path_size; /* length of the search path */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ * and null-terminated role of length role_size
+ * and null-terminated search_path of length search_path_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
+ size_t size, bool transactional);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 8e07bb7409..ea0f7e5cb9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -28,6 +28,7 @@ extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void sequence_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index a16bebf76c..f0fd1ea895 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -88,6 +88,19 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
Size message_size,
const char *message);
+/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Called for the generic logical decoding sequences.
*/
@@ -211,6 +224,20 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
Size message_size,
const char *message);
+/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Called for the streaming generic logical decoding sequences from in-progress
* transactions.
@@ -244,6 +271,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeSequenceCB sequence_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -263,6 +291,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamSequenceCB stream_sequence_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 0bcc150b33..0819361d25 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -131,6 +132,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ char *role;
+ char *search_path;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -438,6 +449,17 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* sequence callback signature */
typedef void (*ReorderBufferSequenceCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn,
@@ -513,6 +535,18 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* stream sequence callback signature */
typedef void (*ReorderBufferStreamSequenceCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn,
@@ -573,6 +607,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
ReorderBufferSequenceCB sequence;
/*
@@ -593,6 +628,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamSequenceCB stream_sequence;
ReorderBufferStreamTruncateCB stream_truncate;
@@ -669,6 +705,9 @@ void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size message_size, const char *message);
void ReorderBufferQueueSequence(ReorderBuffer *rb, TransactionId xid,
Snapshot snapshot, XLogRecPtr lsn, RepOriginId origin_id,
RelFileNode rnode, bool transactional, bool created,
[application/octet-stream] 0003-pgoutput-worker-ddl_replication.patch (33.7K, ../../CAAD30UKTp87+kvGZYL3M2Suxq=WEvFUG24ZRT0yT9rqdkP=uMA@mail.gmail.com/4-0003-pgoutput-worker-ddl_replication.patch)
download | inline diff:
commit 1ada981927a841d433085a42e0c6a7f2cf9cde8d
Author: Zheng (Zane) Li <zhelli@amazon.com>
Date: Fri Mar 18 00:06:40 2022 +0000
0003-pgoutput-worker-ddl_replication.patch
4. Integration with pgoutput
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced
for this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
5. Logical replication worker change
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to
SUBREL_STATE_INIT, So that DML replication can progress on this
new table without manually running
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
6. TAP test
A new TAP test 030_rep_ddl.pl is added. We mainly focused on
testing the happy path of database level replication so far.
Corner case DDLs and table level DDL replication are still to be
carefully tested.
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index c9b0eeefd7..762b897546 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -626,8 +626,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -648,6 +648,63 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
pq_sendbytes(out, message, sz);
}
+/*
+ * Read DDL MESSAGE from stream
+ */
+const char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ const char **role,
+ const char **search_path,
+ bool *transactional,
+ Size *sz)
+{
+ uint8 flags;
+ const char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *role = pq_getmsgstring(in);
+ *search_path = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* encode and send message flags */
+ if (transactional)
+ flags |= MESSAGE_TRANSACTIONAL;
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendstring(out, role);
+ pq_sendstring(out, search_path);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
/*
* Write relation description to the output stream.
*/
@@ -1185,6 +1242,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 03e069c7cd..26fa58973f 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -155,6 +155,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,6 +180,8 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "tcop/pquery.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -329,6 +332,10 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
+static void apply_execute_sql_command(const char *cmdstr,
+ const char* role,
+ const char* search_path,
+ bool isTopLevel);
/* Compute GID for two_phase transactions */
static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
@@ -2360,6 +2367,259 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/*
+ * Handle generic messages.
+ */
+static void
+apply_handle_ddlmessage(StringInfo s)
+{
+ XLogRecPtr lsn;
+ bool transactional;
+ Size sz;
+ const char *prefix;
+ const char *role;
+ const char *search_path;
+ const char *msg;
+
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+
+ apply_execute_sql_command(msg, role, search_path, true);
+}
+
+/*
+ * Add context to the errors produced by apply_execute_sql_command().
+ */
+static void
+execute_sql_command_error_cb(void *arg)
+{
+ errcontext("during execution of SQL statement: %s", (char *) arg);
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE stmt only */
+ char *relname = NULL; /* For CREATE TABLE stmt only */
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ /*
+ * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, CreateStmt may have been freeed up.
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ RangeVar *rv = cstmt->relation;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+ }
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * OK to analyze, rewrite, and plan this query.
+ *
+ * Switch to appropriate context for constructing query and plan trees
+ * (these can't be in the transaction context, as that will get reset
+ * when the command is COMMIT/ROLLBACK). If we have multiple
+ * parsetrees, we use a separate context for each one, so that we can
+ * free that memory before moving on to the next one. But for the
+ * last (or only) parsetree, just use MessageContext, which will be
+ * reset shortly after completion anyway. In event of an error, the
+ * per_parsetree_context will be deleted when MessageContext is reset.
+ */
+ if (lnext(parsetree_list, parsetree_item) != NULL)
+ {
+ per_parsetree_context =
+ AllocSetContextCreate(MessageContext,
+ "per-parsetree message context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+ }
+ else
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(
+ command,
+ cmdstr,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(
+ querytree_list, cmdstr, 0, NULL);
+
+ /*
+ * Done with the snapshot used for parsing/planning.
+ *
+ * While it looks promising to reuse the same snapshot for query
+ * execution (at least for simple protocol), unfortunately it causes
+ * execution to use a snapshot that has been acquired before locking
+ * any of the tables mentioned in the query. This creates user-
+ * visible anomalies, so refrain. Refer to
+ * https://postgr.es/m/flat/5075D8DF.6050500@fuzzy.cz for details.
+ */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in MessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ cmdstr,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ isTopLevel,
+ true,
+ receiver,
+ receiver,
+ NULL);
+ (*receiver->rDestroy) (receiver);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set
+ * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * new table can be replicated without having to manually run
+ * "alter subscription ... refresh publication"
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+ }
+
+ /*
+ * Restore the GUC variables we set above.
+ */
+ AtEOXact_GUC(true, save_nestlevel);
+
+ /* protect against stack resets during CONCURRENTLY processing */
+ if (error_context_stack == &errcallback)
+ error_context_stack = errcallback.previous;
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
/*
* Logical replication protocol message dispatcher.
@@ -2425,6 +2685,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddlmessage(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 5fddab3a3d..ae9c92af2c 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -53,6 +53,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -208,6 +212,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -224,6 +229,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1413,8 +1419,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1440,6 +1446,57 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char * role,
+ const char *search_path, Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+ ListCell *lc;
+
+ /* Reload publications if needed before use. */
+ if (!publications_valid)
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ if (data->publications)
+ list_free_deep(data->publications);
+
+ data->publications = LoadPublications(data->publication_names);
+ MemoryContextSwitchTo(oldctx);
+ publications_valid = true;
+ }
+
+ /* Check if ddl replication is turned on for the publications */
+ foreach(lc, data->publications)
+ {
+ Publication *pub = (Publication *) lfirst(lc);
+ /* TODO need to check relid for table level DDLs */
+ if (!pub->pubactions.pubddl_database && !pub->pubactions.pubddl_table)
+ return;
+ }
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ transactional,
+ prefix,
+ role,
+ search_path,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1725,7 +1782,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl_database =
+ entry->pubactions.pubddl_table = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -1780,6 +1839,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl_database = false;
+ entry->pubactions.pubddl_table = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -1889,6 +1950,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl_database |= pub->pubactions.pubddl_database;
+ entry->pubactions.pubddl_table |= pub->pubactions.pubddl_table;
/*
* We want to publish the changes as the top-most ancestor
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 4d2c881644..862ed467a6 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,14 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
+extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
+ const char **role, const char **search_path,
+ bool *transactional, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index cf61fc1e0f..698c5114e6 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -33,7 +33,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index c924ff35f7..6c9b055eb0 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -23,7 +23,7 @@ $node_subscriber->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index 67b4026afa..2902e6fc34 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -32,7 +32,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 1ce696d4a4..6c586877a1 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -19,7 +19,7 @@ $node_subscriber->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 66e63e755e..de967eaaf2 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -25,9 +25,9 @@ $node_subscriber2->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
# publisher
-$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1");
+$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1 WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_all FOR ALL TABLES");
+ "CREATE PUBLICATION pub_all FOR ALL TABLES WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
"CREATE TABLE tab1 (a int PRIMARY KEY, b text) PARTITION BY LIST (a)");
$node_publisher->safe_psql('postgres',
@@ -424,12 +424,12 @@ $node_publisher->safe_psql('postgres',
# and child tables are present but changes will be replicated via the parent's
# identity and only once.
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# for tab4, we publish changes through the "middle" partitioned table
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# prepare data for the initial sync
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
new file mode 100644
index 0000000000..c88c4ea1c0
--- /dev/null
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -0,0 +1,237 @@
+
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $node_subscriber2 = PostgreSQL::Test::Cluster->new('subscriber2');
+$node_subscriber2->init(allows_streaming => 'logical');
+$node_subscriber2->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber2->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+$node_subscriber2->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+# mypub has pubddl_database on
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+# mypub2 has pubddl_database off
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub2 FOR ALL TABLES with (ddl = '');");
+$node_subscriber2->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub2 CONNECTION '$publisher_connstr' PUBLICATION mypub2;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test smae simple CREATE TABLE command is not replicated to subscriber2 (ddl off)
+# Test ALTER TABLE command is replicated on table test_rep
+# Test CREATE INDEX is replicated to subscriber
+# Test CREATE FUNCTION command is replicated to subscriber
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+$node_publisher->safe_psql('postgres', "CREATE INDEX nameindex on test_rep (name)");
+$node_publisher->safe_psql('postgres', qq{CREATE OR REPLACE FUNCTION totalRecords()
+RETURNS integer AS \$total\$
+declare
+ total integer;
+BEGIN
+ SELECT count(*) into total FROM test_rep;
+ RETURN total;
+END;
+\$total\$ LANGUAGE plpgsql;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't1';");
+is($result, qq(0), 'CREATE of t1 is not replicated to subscriber2');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(1), 'CREATE INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT totalRecords();");
+is($result, qq(2), 'CREATE of function totalRecords replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'CREATE FUNCTION totalrecords is not replicated to subscriber2');
+
+# Test ALTER TABLE DROP
+# Test DROP INDEX
+# Test DROP FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->safe_psql('postgres', "DROP INDEX nameindex;");
+$node_publisher->safe_psql('postgres', "DROP FUNCTION totalRecords;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(0), 'DROP INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'DROP FUNCTION totalrecords replicated');
+
+
+# TODO figure out how to set ON_ERROR_STOP = 0 in this test
+# Test failed CREATE/ALTER TABLE on publisher doesn't break replication
+# Table t1 already exits so expect the command to fail
+#$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+#$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (103, 'data103', 1013);");
+
+#$node_publisher->wait_for_catchup('mysub');
+# Verify replication still works
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+#is($result, qq(1), 'DELETE from test_rep replicated');
+
+# Test DDLs inside txn block
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE t2 (a int, b varchar);
+ALTER TABLE test_rep ADD c3 int;
+INSERT INTO test_rep VALUES (3, 'data3', 3);
+COMMIT;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t2;");
+is($result, qq(0), 'CREATE t2 replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep;");
+is($result, qq(2), 'ALTER test_rep ADD replicated');
+
+# Test toggling pubddl_database option off
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = '');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(f|f), 'pubddl_database turned off on mypub');
+$node_publisher->safe_psql('postgres', "CREATE TABLE t3 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't3';");
+is($result, qq(0), 'CREATE t3 is not replicated');
+
+# Test toggling pubddl_database option on
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = 'database');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(t|t), 'pubddl_database turned on on mypub');
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# A somewhat complicated test in plpgsql block with trigger
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE foo (a int);
+CREATE INDEX foo_idx ON foo (a);
+ALTER TABLE foo ADD COLUMN b timestamptz;
+CREATE FUNCTION foo_ts()
+RETURNS trigger AS $$
+BEGIN
+NEW.b := current_timestamp;
+RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
+FOR EACH ROW EXECUTE FUNCTION foo_ts();
+INSERT INTO foo VALUES (1);
+COMMIT;});
+$result = $node_publisher->safe_psql('postgres', "SELECT b from foo where a = 1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result_sub = $node_subscriber->safe_psql('postgres', "SELECT b from foo where a = 1;");
+is($result, qq($result_sub), 'timestamp of insert matches');
+
+# Test CREATE SCHEMA stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE SCHEMA s1");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_namespace WHERE nspname = 's1';");
+is($result, qq(1), 'CREATE SCHEMA s1 is replicated');
+
+# Test CREATE TABLE in new schema s1 followed by insert
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (2, 'b');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t1;");
+is($result, qq(2), 'CREATE TABLE s1.t1 is replicated');
+
+# Test replication works as expected with mismatched search_path on publisher and subscriber
+$node_publisher->append_conf('postgresql.conf', 'search_path = \'s1, public\'');
+$node_publisher->restart;
+# CREATE unqualified table t2, it is s1.t2 under the modified search_path
+$node_publisher->safe_psql('postgres', "CREATE TABLE t2 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (2, 'b');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (3, 'c');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t2;");
+is($result, qq(3), 'CREATE TABLE s1.t2 is replicated');
+
+# Test owner of new table on subscriber matches the owner on publisher
+$node_publisher->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_subscriber->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_publisher->safe_psql('postgres', "SET SESSION AUTHORIZATION 'ddl_replication_user'; CREATE TABLE t5 (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
+is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+
+#TODO TEST certain DDLs are not replicated
+
+pass "DDL replication tests passed!";
+
+$node_subscriber->stop;
+$node_subscriber2->stop;
+$node_publisher->stop;
+
+done_testing();
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 00:22 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Japin Li @ 2022-03-18 00:22 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Fri, 18 Mar 2022 at 00:22, Zheng Li <zhengli10@gmail.com> wrote:
> Hello Japin,
>>The new patch change the behavior of publication, when we drop a table
>>on publisher, the table also be dropped on subscriber, and this made the
>>regression testa failed, since the regression test will try to drop the
>>table on subscriber.
>
>>IMO, we can disable the DDLs replication by default, which makes the
>>regression test happy. Any thoughts?
>
> Good catch, I forgot to run the whole TAP test suite. I think for now
> we can simply disable DDL replication for the failing tests when
> creating publication: $node_publisher->safe_psql('postgres',
> "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl='')");
> I'll fix it in the next version of patch. I'll also work on breaking
> down the patch into smaller pieces for ease of review.
>
Oh, it doesn't work.
postgres=# CREATE PUBLICATION s1 FOR ALL TABLES WITH(ddl = '');;
ERROR: conflicting or redundant options
Here is the code, I think, you might mean `if (ddl_level_given == NULL)`.
+ if (*ddl_level_given)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options")));
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 00:25 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Japin Li @ 2022-03-18 00:25 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, 18 Mar 2022 at 08:18, Zheng Li <zhengli10@gmail.com> wrote:
> Hello,
>
> Attached please find the broken down patch set. Also fixed the failing
> TAP tests Japin reported.
>
Thanks for updating the patchset, I will try it later.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 00:38 Japin Li <japinli@hotmail.com>
parent: Japin Li <japinli@hotmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Japin Li @ 2022-03-18 00:38 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com; pgsql-hackers@lists.postgresql.org
On Fri, 18 Mar 2022 at 08:22, Japin Li <japinli@hotmail.com> wrote:
> On Fri, 18 Mar 2022 at 00:22, Zheng Li <zhengli10@gmail.com> wrote:
>> Hello Japin,
>>>The new patch change the behavior of publication, when we drop a table
>>>on publisher, the table also be dropped on subscriber, and this made the
>>>regression testa failed, since the regression test will try to drop the
>>>table on subscriber.
>>
>>>IMO, we can disable the DDLs replication by default, which makes the
>>>regression test happy. Any thoughts?
>>
>> Good catch, I forgot to run the whole TAP test suite. I think for now
>> we can simply disable DDL replication for the failing tests when
>> creating publication: $node_publisher->safe_psql('postgres',
>> "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl='')");
>> I'll fix it in the next version of patch. I'll also work on breaking
>> down the patch into smaller pieces for ease of review.
>>
>
> Oh, it doesn't work.
>
> postgres=# CREATE PUBLICATION s1 FOR ALL TABLES WITH(ddl = '');;
> ERROR: conflicting or redundant options
>
>
> Here is the code, I think, you might mean `if (ddl_level_given == NULL)`.
>
> + if (*ddl_level_given)
> + ereport(ERROR,
> + (errcode(ERRCODE_SYNTAX_ERROR),
> + errmsg("conflicting or redundant options")));
> +
> + /*
> + * If publish option was given only the explicitly listed actions
> + * should be published.
> + */
> + pubactions->pubddl_database = false;
> + pubactions->pubddl_table = false;
> +
> + *ddl_level_given = true;
Oh, Sorry, I misunderstand it, it just like you forget initialize
*ddl_level_given to false when enter parse_publication_options().
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 10:19 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Japin Li @ 2022-03-18 10:19 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, 18 Mar 2022 at 08:18, Zheng Li <zhengli10@gmail.com> wrote:
> Hello,
>
> Attached please find the broken down patch set. Also fixed the failing
> TAP tests Japin reported.
>
Here are some comments for the new patches:
Seems like you forget initializing the *ddl_level_given after entering the
parse_publication_options(), see [1].
+ if (*ddl_level_given)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("conflicting or redundant options")));
We can use the errorConflictingDefElem() to replace the ereport() to make the
error message more readable.
I also think that ddl = '' isn't a good way to disable DDL replication, how
about using ddl = 'none' or something else?
The test_decoding test case cannot work as expected, see below:
diff -U3 /home/px/Codes/postgresql/contrib/test_decoding/expected/ddlmessages.out /home/px/Codes/postgresql/Debug/contrib/test_decoding/results/ddlmessages.out
--- /home/px/Codes/postgresql/contrib/test_decoding/expected/ddlmessages.out 2022-03-18 08:46:57.653922671 +0800
+++ /home/px/Codes/postgresql/Debug/contrib/test_decoding/results/ddlmessages.out 2022-03-18 17:34:33.411563601 +0800
@@ -25,8 +25,8 @@
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE tab1;
DROP publication mypub;
- data
----------------------------------------------------------------------------------------------------------------------------------------------------
+ data
+-----------------------------------------------------------------------------------------------------------------------------------------------
DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 47 content:CREATE TABLE tab1 (id serial unique, data int);
BEGIN
sequence public.tab1_id_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
Since the DDL message contains the username, and we try to replace the username with 'redacted' to avoid this problem,
\o | sed 's/role.*search_path/role: redacted, search_path/g'
However, the title and dash lines may have different lengths for different
usernames. To avoid this problem, how about using a specified username when
initializing the database for this regression test?
I try to disable the ddlmessage regression test,
make[2]: Entering directory '/home/px/Codes/postgresql/Debug/src/bin/pg_dump'
rm -rf '/home/px/Codes/postgresql/Debug/src/bin/pg_dump'/tmp_check && /usr/bin/mkdir -p '/home/px/Codes/postgresql/Debug/src/bin/pg_dump'/tmp_check && cd /home/px/Codes/postgresql/Debug/../src/bin/pg_dump
&& TESTDIR='/home/px/Codes/postgresql/Debug/src/bin/pg_dump' PATH="/home/px/Codes/postgresql/Debug/tmp_install/home/px/Codes/postgresql/Debug/pg/bin:/home/px/Codes/postgresql/Debug/src/bin/pg_dump:$PATH"
LD_LIBRARY_PATH="/home/px/Codes/postgresql/Debug/tmp_install/home/px/Codes/postgresql/Debug/pg/lib:$LD_LIBRARY_PATH" PGPORT='65432' PG_REGRESS='/home/px/Codes/postgresql/Debug/src/bin/pg_dump/../../../s
rc/test/regress/pg_regress' /usr/bin/prove -I /home/px/Codes/postgresql/Debug/../src/test/perl/ -I /home/px/Codes/postgresql/Debug/../src/bin/pg_dump t/*.pl
t/001_basic.pl ................ ok
t/002_pg_dump.pl .............. 13/?
# Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub1'
# at t/002_pg_dump.pl line 3916.
# Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
# Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub2'
# at t/002_pg_dump.pl line 3916.
# Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
# Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub3'
# at t/002_pg_dump.pl line 3916.
# Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
# Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub4'
# at t/002_pg_dump.pl line 3916.
# Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
t/002_pg_dump.pl .............. 240/?
[...]
# Review section_post_data results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
t/002_pg_dump.pl .............. 7258/? # Looks like you failed 84 tests of 7709.
t/002_pg_dump.pl .............. Dubious, test returned 84 (wstat 21504, 0x5400)
Failed 84/7709 subtests
t/003_pg_dump_with_server.pl .. ok
t/010_dump_connstr.pl ......... ok
Test Summary Report
-------------------
t/002_pg_dump.pl (Wstat: 21504 Tests: 7709 Failed: 84)
Failed tests: 136-139, 362-365, 588-591, 1040-1043, 1492-1495
1719-1722, 1946-1949, 2177-2180, 2407-2410
2633-2636, 2859-2862, 3085-3088, 3537-3540
3763-3766, 3989-3992, 4215-4218, 4441-4444
5119-5122, 5345-5348, 6702-6705, 7154-7157
Non-zero exit status: 84
Files=4, Tests=7808, 26 wallclock secs ( 0.46 usr 0.05 sys + 7.98 cusr 2.80 csys = 11.29 CPU)
Result: FAIL
make[2]: *** [Makefile:55: check] Error 1
make[2]: Leaving directory '/home/px/Codes/postgresql/Debug/src/bin/pg_dump'
make[1]: *** [Makefile:43: check-pg_dump-recurse] Error 2
make[1]: Leaving directory '/home/px/Codes/postgresql/Debug/src/bin'
make: *** [GNUmakefile:71: check-world-src/bin-recurse] Error 2
And, when I try to use git am to apply the patch, it complains:
$ git am ~/0001-syntax-pg_publication-pg_dump-ddl_replication.patch
Patch format detection failed.
[1] https://www.postgresql.org/message-id/MEYP282MB1669DDF788C623B7F8B64C2EB6139%40MEYP282MB1669.AUSP282...
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 17:25 Zheng Li <zhengli10@gmail.com>
parent: Japin Li <japinli@hotmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-03-18 17:25 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hello,
Thanks for the quick review!
> And, when I try to use git am to apply the patch, it complains:
>
> $ git am ~/0001-syntax-pg_publication-pg_dump-ddl_replication.patch
> Patch format detection failed.
git apply works for me. I'm not sure why git am complains.
I also created a git branch to make code sharing easier, please try this out:
https://github.com/zli236/postgres/tree/ddl_replication
> Seems like you forget initializing the *ddl_level_given after entering the
> parse_publication_options(), see [1].
>
>
> + if (*ddl_level_given)
> + ereport(ERROR,
> + (errcode(ERRCODE_SYNTAX_ERROR),
> + errmsg("conflicting or redundant options")));
>
> We can use the errorConflictingDefElem() to replace the ereport() to make the
> error message more readable.
Agreed. Fixed in the latest version.
> I also think that ddl = '' isn't a good way to disable DDL replication, how
> about using ddl = 'none' or something else?
The syntax follows the existing convention of the WITH publish option.
For example in CREATE PUBLICATION mypub WITH (publish = '')
it means don't publish any DML change. So I'd leave it as is for now.
> The test_decoding test case cannot work as expected, see below:
.....
> DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 47 content:CREATE TABLE tab1 (id serial unique, data int);
> BEGIN
> sequence public.tab1_id_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
>
> Since the DDL message contains the username, and we try to replace the username with 'redacted' to avoid this problem,
>
> \o | sed 's/role.*search_path/role: redacted, search_path/g'
>
> However, the title and dash lines may have different lengths for different
> usernames. To avoid this problem, how about using a specified username when
> initializing the database for this regression test?
I don't understand the question, do you have an example of when the
test doesn't work? It runs fine for me.
> t/002_pg_dump.pl .............. 13/?
> # Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub1'
> # at t/002_pg_dump.pl line 3916.
> # Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
......
> Failed 84/7709 subtests
> t/003_pg_dump_with_server.pl .. ok
> t/010_dump_connstr.pl ......... ok
>
> Test Summary Report
> -------------------
> t/002_pg_dump.pl (Wstat: 21504 Tests: 7709 Failed: 84)
This is fixed in the latest version. I need to remind myself to run
make check-world in the future.
Regards,
Zheng Li
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 17:29 Zheng Li <zhelli@amazon.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-03-18 17:29 UTC (permalink / raw)
To: pgsql-hackers@lists.postgresql.org; +Cc: Zheng Li <zhelli@amazon.com>
git branch of the change:
https://github.com/zli236/postgres/tree/ddl_replication
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-18 23:33 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Japin Li @ 2022-03-18 23:33 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Sat, 19 Mar 2022 at 01:25, Zheng Li <zhengli10@gmail.com> wrote:
> Hello,
>
> Thanks for the quick review!
>
>> And, when I try to use git am to apply the patch, it complains:
>>
>> $ git am ~/0001-syntax-pg_publication-pg_dump-ddl_replication.patch
>> Patch format detection failed.
>
> git apply works for me. I'm not sure why git am complains.
> I also created a git branch to make code sharing easier, please try this out:
> https://github.com/zli236/postgres/tree/ddl_replication
>
Yeah, I can use git apply, I'm not sure how did you generate the patch.
>> I also think that ddl = '' isn't a good way to disable DDL replication, how
>> about using ddl = 'none' or something else?
>
> The syntax follows the existing convention of the WITH publish option.
> For example in CREATE PUBLICATION mypub WITH (publish = '')
> it means don't publish any DML change. So I'd leave it as is for now.
>
Agreed.
>> The test_decoding test case cannot work as expected, see below:
> .....
>> DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 47 content:CREATE TABLE tab1 (id serial unique, data int);
>> BEGIN
>> sequence public.tab1_id_seq: transactional:1 last_value: 1 log_cnt: 0 is_called:0
>>
>> Since the DDL message contains the username, and we try to replace the username with 'redacted' to avoid this problem,
>>
>> \o | sed 's/role.*search_path/role: redacted, search_path/g'
>>
>> However, the title and dash lines may have different lengths for different
>> usernames. To avoid this problem, how about using a specified username when
>> initializing the database for this regression test?
>
> I don't understand the question, do you have an example of when the
> test doesn't work? It runs fine for me.
>
You should use a different user that has different length from your current one.
For example:
px@localhost$ make check-world
>
>> t/002_pg_dump.pl .............. 13/?
>> # Failed test 'binary_upgrade: should dump CREATE PUBLICATION pub1'
>> # at t/002_pg_dump.pl line 3916.
>> # Review binary_upgrade results in /home/px/Codes/postgresql/Debug/src/bin/pg_dump/tmp_check/tmp_test_jgRI
> ......
>> Failed 84/7709 subtests
>> t/003_pg_dump_with_server.pl .. ok
>> t/010_dump_connstr.pl ......... ok
>>
>> Test Summary Report
>> -------------------
>> t/002_pg_dump.pl (Wstat: 21504 Tests: 7709 Failed: 84)
>
> This is fixed in the latest version. I need to remind myself to run
> make check-world in the future.
>
Thanks for updating the patch.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-21 08:13 Dilip Kumar <dilipbalaut@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
3 siblings, 1 reply; 449+ messages in thread
From: Dilip Kumar @ 2022-03-21 08:13 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Thu, Mar 17, 2022 at 2:47 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hi,
>
> >If you don't mind, would you like to share the POC or the branch for this work?
>
> The POC patch is attached. It currently supports the following functionalities:
Thanks for sharing, I will look into it.
> >In such cases why don't we just log the table creation WAL for DDL
> >instead of a complete statement which creates the table and inserts
> >the tuple? Because we are already WAL logging individual inserts and
> >once you make sure of replicating the table creation I think the exact
> >data insertion on the subscriber side will be taken care of by the
> >insert WALs no?
>
> The table creation WAL and table insert WAL are available. The tricky
> part is how do we break down this command into two parts (a normal
> CREATE TABLE followed by insertions) either from the parsetree or the
> WALs. I’ll have to dig more on this.
I agree that this is a bit tricky, anyway I will also put more thoughts on this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-21 20:56 Zheng Li <zhengli10@gmail.com>
parent: Japin Li <japinli@hotmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-03-21 20:56 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Japin,
> You should use a different user that has different length from your current one.
> For example:
>
> px@localhost$ make check-world
This is fixed in the latest commit:
https://github.com/zli236/postgres/commits/ddl_replication
Thanks,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-23 05:09 Japin Li <japinli@hotmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Japin Li @ 2022-03-23 05:09 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, 22 Mar 2022 at 04:56, Zheng Li <zhengli10@gmail.com> wrote:
> Hi Japin,
>
>> You should use a different user that has different length from your current one.
>> For example:
>>
>> px@localhost$ make check-world
>
> This is fixed in the latest commit:
> https://github.com/zli236/postgres/commits/ddl_replication
>
Thanks for fixing this. I rebase the patchset on current master (383f222119)
and attach here for further review.
--
Regrads,
Japin Li.
ChengDu WenWu Information Technology Co.,Ltd.
Attachments:
[text/x-patch] v3-0001-Define-DDL-replication-levels-via-the-CREATE-PUBL.patch (0B, ../../MEYP282MB166926E46397CBFC113B4A7EB6189@MEYP282MB1669.AUSP282.PROD.OUTLOOK.COM/2-v3-0001-Define-DDL-replication-levels-via-the-CREATE-PUBL.patch)
download
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-24 10:02 Dilip Kumar <dilipbalaut@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Dilip Kumar @ 2022-03-24 10:02 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Mon, Mar 21, 2022 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Mar 17, 2022 at 2:47 AM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > Hi,
> >
> > >If you don't mind, would you like to share the POC or the branch for this work?
> >
> > The POC patch is attached. It currently supports the following functionalities:
>
> Thanks for sharing, I will look into it.
>
> > >In such cases why don't we just log the table creation WAL for DDL
> > >instead of a complete statement which creates the table and inserts
> > >the tuple? Because we are already WAL logging individual inserts and
> > >once you make sure of replicating the table creation I think the exact
> > >data insertion on the subscriber side will be taken care of by the
> > >insert WALs no?
> >
> > The table creation WAL and table insert WAL are available. The tricky
> > part is how do we break down this command into two parts (a normal
> > CREATE TABLE followed by insertions) either from the parsetree or the
> > WALs. I’ll have to dig more on this.
>
> I agree that this is a bit tricky, anyway I will also put more thoughts on this.
I had put some more thought about this, basically, during CTAS we are
generating the CreateStmt inside "create_ctas_internal" and executing
it first before inserting the tuple, so can't we generate the
independent sql just for creating the tuple maybe using deparsing or
something?
Apart from that I have one more question, basically if you are
directly logging the sql query then how you are identifying under
which schema you need to create that table, are you changing the sql
and generating schema-qualified name?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-24 11:36 Dilip Kumar <dilipbalaut@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Dilip Kumar @ 2022-03-24 11:36 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Thu, Mar 24, 2022 at 3:32 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Mon, Mar 21, 2022 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > On Thu, Mar 17, 2022 at 2:47 AM Zheng Li <zhengli10@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > >If you don't mind, would you like to share the POC or the branch for this work?
> > >
> > > The POC patch is attached. It currently supports the following functionalities:
> >
> > Thanks for sharing, I will look into it.
> >
> > > >In such cases why don't we just log the table creation WAL for DDL
> > > >instead of a complete statement which creates the table and inserts
> > > >the tuple? Because we are already WAL logging individual inserts and
> > > >once you make sure of replicating the table creation I think the exact
> > > >data insertion on the subscriber side will be taken care of by the
> > > >insert WALs no?
> > >
> > > The table creation WAL and table insert WAL are available. The tricky
> > > part is how do we break down this command into two parts (a normal
> > > CREATE TABLE followed by insertions) either from the parsetree or the
> > > WALs. I’ll have to dig more on this.
> >
> > I agree that this is a bit tricky, anyway I will also put more thoughts on this.
>
> I had put some more thought about this, basically, during CTAS we are
> generating the CreateStmt inside "create_ctas_internal" and executing
> it first before inserting the tuple, so can't we generate the
> independent sql just for creating the tuple maybe using deparsing or
> something?
>
> Apart from that I have one more question, basically if you are
> directly logging the sql query then how you are identifying under
> which schema you need to create that table, are you changing the sql
> and generating schema-qualified name?
I was going through the patch and it seems you are logging the search
path as well along with the query so I think this will probably work.
I have got one more query while looking into the code. In the below
code snippet you are logging DDL command only if it is a top level
query but there are no comments explaining what sort of queries we
don't want to log. Suppose I am executing a DDL statement inside a PL
then that will not be a top level statement so is your intention to
block that as well or that is an unintentional side effect?
+ /*
+ * Consider logging the DDL command if logical logging is
enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-24 17:54 Zheng Li <zhengli10@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-03-24 17:54 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
Hi Dilip,
Thanks for the feedback.
> > > > The table creation WAL and table insert WAL are available. The tricky
> > > > part is how do we break down this command into two parts (a normal
> > > > CREATE TABLE followed by insertions) either from the parsetree or the
> > > > WALs. I’ll have to dig more on this.
> > I had put some more thought about this, basically, during CTAS we are
> > generating the CreateStmt inside "create_ctas_internal" and executing
> > it first before inserting the tuple, so can't we generate the
> > independent sql just for creating the tuple maybe using deparsing or
> > something?
Yes, deparsing might help for edge cases like this. However I found
a simple solution for this specific case:
The idea is to force skipping any direct data population (which can
potentially cause data inconsistency on the subscriber)
in CREATE AS and SELECT INTO command on the subscriber by forcing the
skipData flag in the intoClause of the parsetree after
the logical replication worker parses the command. The data sync will
be taken care of by the DML replication after the DDL replication
finishes.
This is implemented in the latest commit:
https://github.com/zli236/postgres/commit/116c33451da8d44577b8d6fdb05c4b6998cd0167
> > Apart from that I have one more question, basically if you are
> > directly logging the sql query then how you are identifying under
> > which schema you need to create that table, are you changing the sql
> > and generating schema-qualified name?
>
> I was going through the patch and it seems you are logging the search
> path as well along with the query so I think this will probably work.
Yes, currently we log the search path as well as the user name. And we
enforce the same search path and user name when applying the DDL command
on the subscriber.
> I have got one more query while looking into the code. In the below
> code snippet you are logging DDL command only if it is a top level
> query but there are no comments explaining what sort of queries we
> don't want to log. Suppose I am executing a DDL statement inside a PL
> then that will not be a top level statement so is your intention to
> block that as well or that is an unintentional side effect?
>
> + /*
> + * Consider logging the DDL command if logical logging is
> enabled and this is
> + * a top level query.
> + */
> + if (XLogLogicalInfoActive() && isTopLevel)
> + LogLogicalDDLCommand(parsetree, queryString);
Good catch. The reason for having isTopLevel in the condition is
because I haven't decided if a DDL statement inside a PL should
be replicated from the user point of view. For example, if I execute a
plpgsql function or a stored procedure which creates a table under the hood,
does it always make sense to replicate the DDL without running the same
function or stored procedure on the subscriber? It probably depends on
the specific
use case. Maybe we can consider making this behavior configurable by the user.
Thanks,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-03-29 04:17 Dilip Kumar <dilipbalaut@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 3 replies; 449+ messages in thread
From: Dilip Kumar @ 2022-03-29 04:17 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Thu, Mar 24, 2022 at 11:24 PM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hi Dilip,
>
> Thanks for the feedback.
>
> > > > > The table creation WAL and table insert WAL are available. The tricky
> > > > > part is how do we break down this command into two parts (a normal
> > > > > CREATE TABLE followed by insertions) either from the parsetree or the
> > > > > WALs. I’ll have to dig more on this.
>
> > > I had put some more thought about this, basically, during CTAS we are
> > > generating the CreateStmt inside "create_ctas_internal" and executing
> > > it first before inserting the tuple, so can't we generate the
> > > independent sql just for creating the tuple maybe using deparsing or
> > > something?
>
> Yes, deparsing might help for edge cases like this. However I found
> a simple solution for this specific case:
>
> The idea is to force skipping any direct data population (which can
> potentially cause data inconsistency on the subscriber)
> in CREATE AS and SELECT INTO command on the subscriber by forcing the
> skipData flag in the intoClause of the parsetree after
> the logical replication worker parses the command. The data sync will
> be taken care of by the DML replication after the DDL replication
> finishes.
Okay, something like that should work, I am not sure it is the best
design though.
> This is implemented in the latest commit:
> https://github.com/zli236/postgres/commit/116c33451da8d44577b8d6fdb05c4b6998cd0167
>
> > > Apart from that I have one more question, basically if you are
> > > directly logging the sql query then how you are identifying under
> > > which schema you need to create that table, are you changing the sql
> > > and generating schema-qualified name?
> >
> > I was going through the patch and it seems you are logging the search
> > path as well along with the query so I think this will probably work.
>
> Yes, currently we log the search path as well as the user name. And we
> enforce the same search path and user name when applying the DDL command
> on the subscriber.
Yeah this looks fine to me.
>
> > I have got one more query while looking into the code. In the below
> > code snippet you are logging DDL command only if it is a top level
> > query but there are no comments explaining what sort of queries we
> > don't want to log. Suppose I am executing a DDL statement inside a PL
> > then that will not be a top level statement so is your intention to
> > block that as well or that is an unintentional side effect?
> >
> > + /*
> > + * Consider logging the DDL command if logical logging is
> > enabled and this is
> > + * a top level query.
> > + */
> > + if (XLogLogicalInfoActive() && isTopLevel)
> > + LogLogicalDDLCommand(parsetree, queryString);
>
> Good catch. The reason for having isTopLevel in the condition is
> because I haven't decided if a DDL statement inside a PL should
> be replicated from the user point of view. For example, if I execute a
> plpgsql function or a stored procedure which creates a table under the hood,
> does it always make sense to replicate the DDL without running the same
> function or stored procedure on the subscriber? It probably depends on
> the specific
> use case. Maybe we can consider making this behavior configurable by the user.
But then this could be true for DML as well right? Like after
replicating the function to the subscriber if we are sending the DML
done by function then what's the problem in DDL. I mean if there is
no design issue in implementing this then I don't think there is much
point in blocking the same or even providing configuration for this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-06 11:37 Ajin Cherian <itsajin@gmail.com>
parent: Japin Li <japinli@hotmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Ajin Cherian @ 2022-04-06 11:37 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Mar 23, 2022 at 4:09 PM Japin Li <japinli@hotmail.com> wrote:
>
>
> On Tue, 22 Mar 2022 at 04:56, Zheng Li <zhengli10@gmail.com> wrote:
> > Hi Japin,
> >
> >> You should use a different user that has different length from your current one.
> >> For example:
> >>
> >> px@localhost$ make check-world
> >
> > This is fixed in the latest commit:
> > https://github.com/zli236/postgres/commits/ddl_replication
> >
>
> Thanks for fixing this. I rebase the patchset on current master (383f222119)
> and attach here for further review.
>
The patch no longer applies. The patch is a very good attempt, and I
would also like to contribute if required.
I have a few comments but will hold it till a rebased version is available.
regards,
Ajin Cherian
Fujitsu Australia
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-07 10:16 Amit Kapila <amit.kapila16@gmail.com>
parent: Japin Li <japinli@hotmail.com>
1 sibling, 2 replies; 449+ messages in thread
From: Amit Kapila @ 2022-04-07 10:16 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Mar 23, 2022 at 10:39 AM Japin Li <japinli@hotmail.com> wrote:
>
> Thanks for fixing this. I rebase the patchset on current master (383f222119)
> and attach here for further review.
>
Some initial comments:
===================
1.
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
I don't see anywhere the patch using a non-transactional message. Is
it required for any future usage? The non-transactional behavior has
some known consistency issues, see email [1].
2. For DDL replication, do we need to wait for a consistent point of
snapshot? For DMLs, that point is a convenient point to initialize
replication from, which is why we export a snapshot at that point,
which is used to read normal data. Do we have any similar needs for
DDL replication?
3. The patch seems to be creating an entry in pg_subscription_rel for
'create' message. Do we need some handling on Drop, if not, why? I
think adding some comments for this aspect would make it easier to
follow.
4. The handling related to partition tables seems missing because, on
the subscriber-side, it always creates a relation entry in
pg_subscription_rel which won't work. Check its interaction with
publish_via_partition_root.
[1] - https://www.postgresql.org/message-id/CAA4eK1KAFdQEULk%2B4C%3DieWA5UPSUtf_gtqKsFj9J9f2c%3D8hm4g%40ma...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-07 10:19 Amit Kapila <amit.kapila16@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
2 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-04-07 10:19 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Tue, Mar 29, 2022 at 9:47 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Thu, Mar 24, 2022 at 11:24 PM Zheng Li <zhengli10@gmail.com> wrote:
> >
> >
> > Good catch. The reason for having isTopLevel in the condition is
> > because I haven't decided if a DDL statement inside a PL should
> > be replicated from the user point of view. For example, if I execute a
> > plpgsql function or a stored procedure which creates a table under the hood,
> > does it always make sense to replicate the DDL without running the same
> > function or stored procedure on the subscriber? It probably depends on
> > the specific
> > use case. Maybe we can consider making this behavior configurable by the user.
>
> But then this could be true for DML as well right? Like after
> replicating the function to the subscriber if we are sending the DML
> done by function then what's the problem in DDL. I mean if there is
> no design issue in implementing this then I don't think there is much
> point in blocking the same or even providing configuration for this.
>
Valid point. I think if there are some design/implementation
constraints for this then it is better to document those(probably as
comments).
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-08 11:00 Amit Kapila <amit.kapila16@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
2 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-04-08 11:00 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Tue, Mar 29, 2022 at 9:47 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > The idea is to force skipping any direct data population (which can
> > potentially cause data inconsistency on the subscriber)
> > in CREATE AS and SELECT INTO command on the subscriber by forcing the
> > skipData flag in the intoClause of the parsetree after
> > the logical replication worker parses the command. The data sync will
> > be taken care of by the DML replication after the DDL replication
> > finishes.
>
> Okay, something like that should work, I am not sure it is the best
> design though.
>
Even if this works, how will we make Alter Table statement work where
it needs to rewrite the table? There also I think we can face a
similar problem if we directly send the statement, once the table will
be updated due to the DDL statement and then again due to table
rewrite as that will have a separate WAL.
Another somewhat unrelated problem I see with this work is how to save
recursion of the same command between nodes (when the involved nodes
replicate DDLs). For DMLs, we can avoid that via replication origins
as is being done in the patch proposed [1] but not sure how will we
deal with that here?
[1] - https://commitfest.postgresql.org/38/3610/
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-08 11:13 Amit Kapila <amit.kapila16@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-04-08 11:13 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Thu, Mar 17, 2022 at 3:36 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> Did you see some old code I wrote towards this goal?
> https://www.postgresql.org/message-id/20150215044814.GL3391@alvh.no-ip.org
> The intention was that DDL would produce some JSON blob that accurately
> describes the DDL that was run;
>
I have read that thread and found one of your emails [1] where you
seem to be saying that JSON representation is not required for BDR.
Will in some way going via JSON blob way make this project
easier/better?
> the caller can acquire that and use it
> to produce working DDL that doesn't depend on runtime conditions.
>
For runtime conditions, one of the things you have mentioned in that
thread is to add schema name in the statement at the required places
which this patch deals with in a different way by explicitly sending
it along with the DDL statement. The other cases where we might need
deparsing are Alter Table type cases (where we need to rewrite the
table) where we may want to send a different DDL. I haven't analyzed
but I think it is better to have a list where all we need deparsing
and what is the best way to deal with it. The simpler cases seem to be
working with the approach proposed by this patch but I am not sure if
it will work for all kinds of cases.
[1] - https://www.postgresql.org/message-id/20150504185721.GB2523%40alvh.no-ip.org
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-08 11:34 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 3 replies; 449+ messages in thread
From: Alvaro Herrera @ 2022-04-08 11:34 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On 2022-Apr-08, Amit Kapila wrote:
> On Thu, Mar 17, 2022 at 3:36 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > Did you see some old code I wrote towards this goal?
> > https://www.postgresql.org/message-id/20150215044814.GL3391@alvh.no-ip.org
> > The intention was that DDL would produce some JSON blob that accurately
> > describes the DDL that was run;
>
> I have read that thread and found one of your emails [1] where you
> seem to be saying that JSON representation is not required for BDR.
> Will in some way going via JSON blob way make this project
> easier/better?
I don't know if replication support will be easier by using JSON; I just
think that JSON makes the overall feature more easily usable for other
purposes.
I am not familiar with BDR replication nowadays.
> For runtime conditions, one of the things you have mentioned in that
> thread is to add schema name in the statement at the required places
> which this patch deals with in a different way by explicitly sending
> it along with the DDL statement.
Hmm, ok. The point of the JSON-blob route is that the publisher sends a
command representation that can be parsed/processed/transformed
arbitrarily by the subscriber using generic rules; it should be trivial
to use a JSON tool to change schema A to schema B in any arbitrary DDL
command, and produce another working DDL command without having to know
how to write that command specifically. So if I have a rule that
"schema A there is schema B here", all DDL commands can be replayed with
no further coding (without having to rely on getting the run-time
search_path correct.)
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"El sudor es la mejor cura para un pensamiento enfermo" (Bardia)
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-08 13:12 Robert Haas <robertmhaas@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 0 replies; 449+ messages in thread
From: Robert Haas @ 2022-04-08 13:12 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Fri, Apr 8, 2022 at 7:34 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > For runtime conditions, one of the things you have mentioned in that
> > thread is to add schema name in the statement at the required places
> > which this patch deals with in a different way by explicitly sending
> > it along with the DDL statement.
>
> Hmm, ok. The point of the JSON-blob route is that the publisher sends a
> command representation that can be parsed/processed/transformed
> arbitrarily by the subscriber using generic rules; it should be trivial
> to use a JSON tool to change schema A to schema B in any arbitrary DDL
> command, and produce another working DDL command without having to know
> how to write that command specifically. So if I have a rule that
> "schema A there is schema B here", all DDL commands can be replayed with
> no further coding (without having to rely on getting the run-time
> search_path correct.)
Yeah, that was a really nice aspect of that approach.
--
Robert Haas
EDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 05:00 Amit Kapila <amit.kapila16@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-04-11 05:00 UTC (permalink / raw)
To: Japin Li <japinli@hotmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Thu, Apr 7, 2022 at 3:46 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Mar 23, 2022 at 10:39 AM Japin Li <japinli@hotmail.com> wrote:
>
> 2. For DDL replication, do we need to wait for a consistent point of
> snapshot? For DMLs, that point is a convenient point to initialize
> replication from, which is why we export a snapshot at that point,
> which is used to read normal data. Do we have any similar needs for
> DDL replication?
>
I have thought a bit more about this and I think we need to build the
snapshot for DML replication as we need to read catalog tables to
decode the corresponding WAL but it is not clear to me if we have a
similar requirement for DDL replication. If the catalog access is
required then it makes sense to follow the current snapshot model,
otherwise, we may need to think differently for DDL replication.
One more related point is that for DML replication, we do ensure that
we copy the entire data of the table (via initial sync) which exists
even before the publication for that table exists, so do we want to do
something similar for DDLs? How do we sync the schema of the table
before the user has defined the publication? Say the table has been
created before the publication is defined and after that, there are
only Alter statements, so do we expect, users to create the table on
the subscriber and then we can replicate the Alter statements? And
even if we do that it won't be clear which Alter statements will be
replicated after publication is defined especially if those Alters
happened concurrently with defining publications?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 05:04 Amit Kapila <amit.kapila16@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-04-11 05:04 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Fri, Apr 8, 2022 at 5:04 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2022-Apr-08, Amit Kapila wrote:
>
>
> > For runtime conditions, one of the things you have mentioned in that
> > thread is to add schema name in the statement at the required places
> > which this patch deals with in a different way by explicitly sending
> > it along with the DDL statement.
>
> Hmm, ok. The point of the JSON-blob route is that the publisher sends a
> command representation that can be parsed/processed/transformed
> arbitrarily by the subscriber using generic rules; it should be trivial
> to use a JSON tool to change schema A to schema B in any arbitrary DDL
> command, and produce another working DDL command without having to know
> how to write that command specifically. So if I have a rule that
> "schema A there is schema B here", all DDL commands can be replayed with
> no further coding (without having to rely on getting the run-time
> search_path correct.)
>
Okay, thanks for the clarification.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 12:46 Euler Taveira <euler@eulerto.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Euler Taveira @ 2022-04-11 12:46 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; japin <japinli@hotmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, "PostgreSQL Hackers" <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 11, 2022, at 2:00 AM, Amit Kapila wrote:
> On Thu, Apr 7, 2022 at 3:46 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Mar 23, 2022 at 10:39 AM Japin Li <japinli@hotmail.com> wrote:
> >
> > 2. For DDL replication, do we need to wait for a consistent point of
> > snapshot? For DMLs, that point is a convenient point to initialize
> > replication from, which is why we export a snapshot at that point,
> > which is used to read normal data. Do we have any similar needs for
> > DDL replication?
> >
>
> I have thought a bit more about this and I think we need to build the
> snapshot for DML replication as we need to read catalog tables to
> decode the corresponding WAL but it is not clear to me if we have a
> similar requirement for DDL replication. If the catalog access is
> required then it makes sense to follow the current snapshot model,
> otherwise, we may need to think differently for DDL replication.
>
> One more related point is that for DML replication, we do ensure that
> we copy the entire data of the table (via initial sync) which exists
> even before the publication for that table exists, so do we want to do
> something similar for DDLs? How do we sync the schema of the table
> before the user has defined the publication? Say the table has been
> created before the publication is defined and after that, there are
> only Alter statements, so do we expect, users to create the table on
> the subscriber and then we can replicate the Alter statements? And
> even if we do that it won't be clear which Alter statements will be
> replicated after publication is defined especially if those Alters
> happened concurrently with defining publications?
The *initial* DDL replication is a different problem than DDL replication. The
former requires a snapshot to read the current catalog data and build a CREATE
command as part of the subscription process. The subsequent DDLs in that object
will be handled by a different approach that is being discussed here.
I'm planning to work on the initial DDL replication. I'll open a new thread as
soon as I write a design for it. Just as an example, the pglogical approach is
to use pg_dump behind the scenes to provide the schema [1]. It is a reasonable
approach but an optimal solution should be an API to provide the initial DDL
commands. I mean the main point of this feature is to have an API to create an
object that the logical replication can use it for initial schema
synchronization. This "DDL to create an object" was already discussed in the
past [2].
[1] https://github.com/2ndQuadrant/pglogical/blob/REL2_x_STABLE/pglogical_sync.c#L942
[2] https://www.postgresql.org/message-id/4E69156E.5060509%40dunslane.net
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 16:42 Zheng Li <zhengli10@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
2 siblings, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-11 16:42 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; Ajin Cherian <itsajin@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
Hi,
> > Good catch. The reason for having isTopLevel in the condition is
> > because I haven't decided if a DDL statement inside a PL should
> > be replicated from the user point of view. For example, if I execute a
> > plpgsql function or a stored procedure which creates a table under the hood,
> > does it always make sense to replicate the DDL without running the same
> > function or stored procedure on the subscriber? It probably depends on
> > the specific
> > use case. Maybe we can consider making this behavior configurable by the user.
>
> But then this could be true for DML as well right? Like after
> replicating the function to the subscriber if we are sending the DML
> done by function then what's the problem in DDL. I mean if there is
> no design issue in implementing this then I don't think there is much
> point in blocking the same or even providing configuration for this.
Agreed. I'll unblock DDLs in functions/procedures and test it out. I
find out some DDLs in functions are replicated multiple times on the
subscriber while they should only be replicated once. Still trying to
figure out why.
>The patch no longer applies. The patch is a very good attempt, and I
>would also like to contribute if required.
>I have a few comments but will hold it till a rebased version is available.
Hi, Ajin. That'll be great! I'll rebase my github branch to master
head. In the meantime, this github branch still works
https://github.com/zli236/postgres/tree/ddl_replication
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 17:31 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-11 17:31 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Amit,
> Some initial comments:
> ===================
> 1.
> +/*
> + * Write logical decoding DDL message into XLog.
> + */
> +XLogRecPtr
> +LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
> + size_t size, bool transactional)
>
> I don't see anywhere the patch using a non-transactional message. Is
> it required for any future usage? The non-transactional behavior has
> some known consistency issues, see email [1].
The transactional flag is not required by the current usage. I thought
it might be useful if other logical decoding plugins want to log and
consume DDL messages in a non-transactional way. But I don't have a
specific use case yet.
> 2. For DDL replication, do we need to wait for a consistent point of
> snapshot? For DMLs, that point is a convenient point to initialize
> replication from, which is why we export a snapshot at that point,
> which is used to read normal data. Do we have any similar needs for
> DDL replication?
The current design requires manual schema initialization on the
subscriber before the logical replication setup.
As Euler Taveira pointed out, snapshot is needed in initial schema
synchronization. And that is a different topic.
> 3. The patch seems to be creating an entry in pg_subscription_rel for
> 'create' message. Do we need some handling on Drop, if not, why? I
> think adding some comments for this aspect would make it easier to
> follow.
It's already handled by existing logic in heap_drop_with_catalog:
https://github.com/zli236/postgres/blob/ddl_replication/src/backend/catalog/heap.c#L2005
I'll add some comment.
> 4. The handling related to partition tables seems missing because, on
> the subscriber-side, it always creates a relation entry in
> pg_subscription_rel which won't work. Check its interaction with
> publish_via_partition_root.
I will test it out.
>Even if this works, how will we make Alter Table statement work where
>it needs to rewrite the table? There also I think we can face a
>similar problem if we directly send the statement, once the table will
>be updated due to the DDL statement and then again due to table
>rewrite as that will have a separate WAL.
Yes, I think any DDL that can generate DML changes should be listed
out and handled properly or documented. Here is one extreme example
involving volatile functions:
ALTER TABLE nd_ddl ADD COLUMN t timestamp DEFAULT now().
Again, I think we need to somehow skip the data rewrite on the
subscriber when replicating such DDL and let DML replication handle
the rewrite.
>Another somewhat unrelated problem I see with this work is how to save
>recursion of the same command between nodes (when the involved nodes
>replicate DDLs). For DMLs, we can avoid that via replication origins
>as is being done in the patch proposed [1] but not sure how will we
>deal with that here?
I'll need to investigate "recursion of the same command between
nodes", could you provide an example?
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-11 17:36 Zheng Li <zhengli10@gmail.com>
parent: Euler Taveira <euler@eulerto.com>
1 sibling, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-11 17:36 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; japin <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> I'm planning to work on the initial DDL replication. I'll open a new thread as
> soon as I write a design for it. Just as an example, the pglogical approach is
> to use pg_dump behind the scenes to provide the schema [1]. It is a reasonable
> approach but an optimal solution should be an API to provide the initial DDL
> commands. I mean the main point of this feature is to have an API to create an
> object that the logical replication can use it for initial schema
> synchronization. This "DDL to create an object" was already discussed in the
> past [2].
Nice! I think automatic initial schema synchronization for replication
is a very useful feature.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-12 10:55 Amit Kapila <amit.kapila16@gmail.com>
parent: Euler Taveira <euler@eulerto.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-04-12 10:55 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: japin <japinli@hotmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 11, 2022 at 6:16 PM Euler Taveira <euler@eulerto.com> wrote:
>
> On Mon, Apr 11, 2022, at 2:00 AM, Amit Kapila wrote:
>
> On Thu, Apr 7, 2022 at 3:46 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Mar 23, 2022 at 10:39 AM Japin Li <japinli@hotmail.com> wrote:
> >
> > 2. For DDL replication, do we need to wait for a consistent point of
> > snapshot? For DMLs, that point is a convenient point to initialize
> > replication from, which is why we export a snapshot at that point,
> > which is used to read normal data. Do we have any similar needs for
> > DDL replication?
> >
>
> I have thought a bit more about this and I think we need to build the
> snapshot for DML replication as we need to read catalog tables to
> decode the corresponding WAL but it is not clear to me if we have a
> similar requirement for DDL replication. If the catalog access is
> required then it makes sense to follow the current snapshot model,
> otherwise, we may need to think differently for DDL replication.
>
> One more related point is that for DML replication, we do ensure that
> we copy the entire data of the table (via initial sync) which exists
> even before the publication for that table exists, so do we want to do
> something similar for DDLs? How do we sync the schema of the table
> before the user has defined the publication? Say the table has been
> created before the publication is defined and after that, there are
> only Alter statements, so do we expect, users to create the table on
> the subscriber and then we can replicate the Alter statements? And
> even if we do that it won't be clear which Alter statements will be
> replicated after publication is defined especially if those Alters
> happened concurrently with defining publications?
>
> The *initial* DDL replication is a different problem than DDL replication. The
> former requires a snapshot to read the current catalog data and build a CREATE
> command as part of the subscription process. The subsequent DDLs in that object
> will be handled by a different approach that is being discussed here.
>
I think they are not completely independent because of the current way
to do initial sync followed by replication. The initial sync and
replication need some mechanism to ensure that one of those doesn't
overwrite the work done by the other. Now, the initial idea and patch
can be developed separately but I think both the patches have some
dependency.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-12 11:12 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-04-12 11:12 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Mon, Apr 11, 2022 at 11:01 PM Zheng Li <zhengli10@gmail.com> wrote:
>
> >Even if this works, how will we make Alter Table statement work where
> >it needs to rewrite the table? There also I think we can face a
> >similar problem if we directly send the statement, once the table will
> >be updated due to the DDL statement and then again due to table
> >rewrite as that will have a separate WAL.
>
> Yes, I think any DDL that can generate DML changes should be listed
> out and handled properly or documented. Here is one extreme example
> involving volatile functions:
> ALTER TABLE nd_ddl ADD COLUMN t timestamp DEFAULT now().
> Again, I think we need to somehow skip the data rewrite on the
> subscriber when replicating such DDL and let DML replication handle
> the rewrite.
>
I am not sure what is the right way to deal with this but another idea
we can investigate is to probably rewrite the DDL such that it doesn't
rewrite the table.
> >Another somewhat unrelated problem I see with this work is how to save
> >recursion of the same command between nodes (when the involved nodes
> >replicate DDLs). For DMLs, we can avoid that via replication origins
> >as is being done in the patch proposed [1] but not sure how will we
> >deal with that here?
>
> I'll need to investigate "recursion of the same command between
> nodes", could you provide an example?
>
See email [1]. I think the same solution should work for your proposal
as well because I see that LogLogicalMessage includes origin but it is
better if you can confirm it.
[1] - https://www.postgresql.org/message-id/flat/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail....
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-13 00:19 Zheng Li <zhengli10@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-04-13 00:19 UTC (permalink / raw)
To: Ajin Cherian <itsajin@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi,
Here is the rebased new branch
https://github.com/zli236/postgres/commits/ddl_replication
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-13 03:47 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-04-13 03:47 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Ajin Cherian <itsajin@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Apr 13, 2022 at 5:49 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hi,
>
> Here is the rebased new branch
> https://github.com/zli236/postgres/commits/ddl_replication
>
Thanks, but it would be good if you can share it in the patch form as
well unless you need to send patches too frequently. It is easier to
give comments.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-13 09:07 Dilip Kumar <dilipbalaut@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Dilip Kumar @ 2022-04-13 09:07 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Euler Taveira <euler@eulerto.com>; japin <japinli@hotmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > The *initial* DDL replication is a different problem than DDL replication. The
> > former requires a snapshot to read the current catalog data and build a CREATE
> > command as part of the subscription process. The subsequent DDLs in that object
> > will be handled by a different approach that is being discussed here.
> >
>
> I think they are not completely independent because of the current way
> to do initial sync followed by replication. The initial sync and
> replication need some mechanism to ensure that one of those doesn't
> overwrite the work done by the other. Now, the initial idea and patch
> can be developed separately but I think both the patches have some
> dependency.
I agree with the point that their design can not be completely
independent. They have some logical relationship of what schema will
be copied by the initial sync and where is the exact boundary from
which we will start sending as replication. And suppose first we only
plan to implement the replication part then how the user will know
what all schema user has to create and what will be replicated using
DDL replication? Suppose the user takes a dump and copies all the
schema and then creates the subscription, then how we are we going to
handle the DDL concurrent to the subscription command?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-13 09:50 Amit Kapila <amit.kapila16@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Amit Kapila @ 2022-04-13 09:50 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Euler Taveira <euler@eulerto.com>; japin <japinli@hotmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Apr 13, 2022 at 2:38 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > > The *initial* DDL replication is a different problem than DDL replication. The
> > > former requires a snapshot to read the current catalog data and build a CREATE
> > > command as part of the subscription process. The subsequent DDLs in that object
> > > will be handled by a different approach that is being discussed here.
> > >
> >
> > I think they are not completely independent because of the current way
> > to do initial sync followed by replication. The initial sync and
> > replication need some mechanism to ensure that one of those doesn't
> > overwrite the work done by the other. Now, the initial idea and patch
> > can be developed separately but I think both the patches have some
> > dependency.
>
> I agree with the point that their design can not be completely
> independent. They have some logical relationship of what schema will
> be copied by the initial sync and where is the exact boundary from
> which we will start sending as replication. And suppose first we only
> plan to implement the replication part then how the user will know
> what all schema user has to create and what will be replicated using
> DDL replication? Suppose the user takes a dump and copies all the
> schema and then creates the subscription, then how we are we going to
> handle the DDL concurrent to the subscription command?
>
Right, I also don't see how it can be done in the current
implementation. So, I think even if we want to develop these two as
separate patches they need to be integrated to make the solution
complete.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-13 17:22 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-13 17:22 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Euler Taveira <euler@eulerto.com>; japin <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
>Thanks, but it would be good if you can share it in the patch form as
>well unless you need to send patches too frequently. It is easier to
>give comments.
The rebased patches are attached. They apply on commit ed0fbc8.
> > I agree with the point that their design can not be completely
> > independent. They have some logical relationship of what schema will
> > be copied by the initial sync and where is the exact boundary from
> > which we will start sending as replication. And suppose first we only
> > plan to implement the replication part then how the user will know
> > what all schema user has to create and what will be replicated using
> > DDL replication? Suppose the user takes a dump and copies all the
> > schema and then creates the subscription, then how we are we going to
> > handle the DDL concurrent to the subscription command?
> >
>
> Right, I also don't see how it can be done in the current
> implementation. So, I think even if we want to develop these two as
> separate patches they need to be integrated to make the solution
> complete.
The concurrent DDL issue actually exists today even with DML
replication. i.e. user have to make sure table schema is in sync
before starting logical replication.
Here is a high level idea to implement schema initialization:
1. On source DB: dump the schema and create a logical replication slot
on a snapshot at the same time
2. On target DB: restore the schema
3. On target DB: create subscription by pointing it to the logical
replication slot and the snapshot in step 1, so that the initial sync
is based on the snapshot and concurrent changes are replicated using
the logical replication slot.
Regards,
Zheng
Attachments:
[application/octet-stream] v4-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch (8.0K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/2-v4-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch)
download | inline diff:
From 08999be2772ffa47fa273070451ab2e6f15afc65 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Thu, 24 Mar 2022 17:46:55 +0000
Subject: [PATCH 5/6] Support replication of CREATE ... AS .. and SELECT ...
INTO ... statements.
The idea is to force skipping the direct data population
(which could potentially cause data mismatch compared to the publisher)
with these command on the subscriber by force setting the skipData flag
in the intoClause of the parsetree after the logical replication worker
parses the command. The data sync will be taken care of by DML replication
after the replication of the DDL command.
---
src/backend/replication/logical/worker.c | 51 ++++++++++++++++++++----
src/backend/tcop/utility.c | 41 +++++++++++--------
src/test/subscription/t/030_rep_ddls.pl | 24 +++++++++++
3 files changed, 92 insertions(+), 24 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f9682dcaac..f5274531b7 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2543,23 +2543,58 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
Portal portal;
DestReceiver *receiver;
bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE stmt only */
- char *relname = NULL; /* For CREATE TABLE stmt only */
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
commandTag = CreateCommandTag((Node *)command);
/*
- * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
- * execute the stmt. At that point, CreateStmt may have been freeed up.
+ * execute the stmt. At that point, command->stmt may have been freeed up.
*/
if (commandTag == CMDTAG_CREATE_TABLE)
{
- CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
}
+ else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ RangeVar *rv = castmt->into->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ /* SELECT INTO */
+ else if (commandTag == CMDTAG_SELECT)
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ RangeVar *rv = sstmt->intoClause->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
@@ -2658,12 +2693,12 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
* Table created by DDL replication (database level) is automatically
* added to the subscription here.
*
- * Call AddSubscriptionRelState for CREATE TABEL command to set
- * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+ * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
* new table can be replicated without having to manually run
* "alter subscription ... refresh publication"
*/
- if (commandTag == CMDTAG_CREATE_TABLE)
+ if (relname != NULL)
{
Oid relid;
Oid relnamespace = InvalidOid;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index b764d9df3d..68d45f3101 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1142,24 +1142,31 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/*
* CreateTableAsStmt can create either a table a materialized view
- * and they are handled differently.
*/
case T_CreateTableAsStmt:
{
CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+
switch(stmt->objtype)
{
+ /*
+ * Either CREATE TABLE AS or SELECT ... INTO stmt
+ * The statement is logged as is, but when we apply the
+ * CREATE TABLE AS or SELECT ... INTO statemtns on the logical
+ * replication worker, we will force the skipData flag in the
+ * intoClause. This way we avoid direct data population on the
+ * subsriber with the execution of these commands which can
+ * potentially cause data mismatch bewteen the publisher.
+ *
+ * The data sync will be handled by DML replication after the
+ * target table has been created.
+ */
case OBJECT_TABLE:
- /*
- * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
- * 1. A normal CREATE TABLE string that get's logged and replicated via
- * DDL replication.
- * 2. Insertions that get replicated by DML replication.
- */
- break;
+
+ /* CREATE MATERIALIZED VIEW */
case OBJECT_MATVIEW:
/*
- * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * Log these stmt for logical replication if
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
@@ -1304,8 +1311,17 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
if (isCompleteQuery)
+ {
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+ }
+
switch (nodeTag(parsetree))
{
/*
@@ -2135,13 +2151,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a complete top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 562efe2cf7..3b15c6d9f0 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -244,6 +244,30 @@ $node_publisher->wait_for_catchup('mysub');
$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+# TEST CREATE TABLE AS stmt
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t3 AS SELECT a, b from s1.t1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t3;");
+is($result, qq(2), 'CREATE TABLE s1.t3 AS is replicated with data');
+
+# TEST CREATE TABLE AS stmt ... WITH NO DATA
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t4 AS SELECT a, b from s1.t1 WITH NO DATA;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t4;");
+is($result, qq(0), 'CREATE TABLE s1.t4 AS is replicated with no data');
+
+# TEST SELECT INTO stmt
+$node_publisher->safe_psql('postgres', "SELECT b into s1.t6 from s1.t1 where a > 1");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
+is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] v4-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch (65.0K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/3-v4-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch)
download | inline diff:
From 3647c81db136aabaed95bc0567b1c5229363f2a0 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:47:16 +0000
Subject: [PATCH 1/6] Define DDL replication levels via the CREATE PUBLICATION
command, currently allow Database or Table level DDL replication.
Allow the user to configure either database level or table level
DDL replication via the CREATE PUBLICATION command as in my first
email. Two new columns are added to the pg_publication catalog to
show the DDL replication levels, test output publication.out is
updated accordingly. pg_dump is also modified to accommodate the
pg_publication catalog change.
---
src/backend/catalog/pg_publication.c | 2 +
src/backend/commands/publicationcmds.c | 89 +++++-
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/pg_dump.c | 45 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +-
src/bin/psql/describe.c | 30 +-
src/include/catalog/pg_publication.h | 10 +
src/include/commands/defrem.h | 1 +
src/test/regress/expected/publication.out | 372 +++++++++++-----------
10 files changed, 364 insertions(+), 199 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 2631558ff1..ddf1363477 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1006,6 +1006,8 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubactions.pubddl_database = pubform->pubddl_database;
+ pub->pubactions.pubddl_table = pubform->pubddl_table;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 6df0e6670f..427110bf23 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -80,15 +80,18 @@ static void PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok);
static void
parse_publication_options(ParseState *pstate,
List *options,
+ bool for_all_tables,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *ddl_level_given)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *ddl_level_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -96,6 +99,16 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ if (for_all_tables)
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ else
+ {
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+ }
/* Parse options */
foreach(lc, options)
@@ -154,6 +167,55 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "ddl") == 0)
+ {
+ char *ddl_level;
+ List *ddl_list;
+ ListCell *lc;
+
+ if (*ddl_level_given)
+ errorConflictingDefElem(defel, pstate);
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
+ ddl_level = defGetString(defel);
+
+ if (!SplitIdentifierString(ddl_level, ',', &ddl_list))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax for \"ddl\" option")));
+
+ /* Process the option list. */
+ foreach(lc, ddl_list)
+ {
+ char *publish_opt = (char *) lfirst(lc);
+
+ if (strcmp(publish_opt, "database") == 0)
+ {
+ if (!for_all_tables)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("DDL replication on the database level is only supported in FOR ALL TABLES publication")));
+ else
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ }
+ else if (strcmp(publish_opt, "table") == 0)
+ pubactions->pubddl_table = true;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized \"ddl\" value: \"%s\"", publish_opt)));
+ }
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -760,6 +822,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Datum values[Natts_pg_publication];
HeapTuple tup;
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -802,9 +865,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
parse_publication_options(pstate,
stmt->options,
+ stmt->for_all_tables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -821,6 +886,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubddl_database - 1] =
+ BoolGetDatum(pubactions.pubddl_database);
+ values[Anum_pg_publication_pubddl_table - 1] =
+ BoolGetDatum(pubactions.pubddl_table);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -907,6 +976,7 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
bool replaces[Natts_pg_publication];
Datum values[Natts_pg_publication];
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -915,11 +985,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
List *root_relids = NIL;
ListCell *lc;
+ pubform = (Form_pg_publication) GETSTRUCT(tup);
+
parse_publication_options(pstate,
stmt->options,
+ pubform->puballtables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -1023,6 +1097,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
+ if (ddl_level_given)
+ {
+ values[Anum_pg_publication_pubddl_database - 1] = BoolGetDatum(pubactions.pubddl_database);
+ replaces[Anum_pg_publication_pubddl_database - 1] = true;
+
+ values[Anum_pg_publication_pubddl_table - 1] = BoolGetDatum(pubactions.pubddl_table);
+ replaces[Anum_pg_publication_pubddl_table - 1] = true;
+ }
+
if (publish_via_partition_root_given)
{
values[Anum_pg_publication_pubviaroot - 1] = BoolGetDatum(publish_via_partition_root);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 43f14c233d..e11d2c03cc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5634,6 +5634,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl_database |= pubform->pubddl_database;
+ pubdesc->pubactions.pubddl_table |= pubform->pubddl_table;
/*
* Check if all columns referenced in the filter expression are part of
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 969e2a7a46..642c222212 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3810,6 +3810,8 @@ getPublications(Archive *fout, int *numPublications)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubddl_database;
+ int i_pubddl_table;
int i,
ntups;
@@ -3824,23 +3826,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubddl_database, p.pubddl_table "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3857,6 +3865,8 @@ getPublications(Archive *fout, int *numPublications)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubddl_database = PQfnumber(res, "pubddl_database");
+ i_pubddl_table = PQfnumber(res, "pubddl_table");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3881,6 +3891,10 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubddl_database =
+ (strcmp(PQgetvalue(res, i, i_pubddl_database), "t") == 0);
+ pubinfo[i].pubddl_table =
+ (strcmp(PQgetvalue(res, i, i_pubddl_table), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -3960,6 +3974,29 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
appendPQExpBufferStr(query, "'");
+ appendPQExpBufferStr(query, ", ddl = '");
+ first = true;
+
+ if (pubinfo->pubddl_database)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "database");
+ first = false;
+ }
+
+ if (pubinfo->pubddl_table)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "table");
+ first = false;
+ }
+
+ appendPQExpBufferStr(query, "'");
+
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..8c5ef01cb0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,8 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index c65c92bfb0..2b8967582e 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2420,7 +2420,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2429,9 +2429,9 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub2
FOR ALL TABLES
- WITH (publish = \'\');',
+ WITH (publish = \'\', ddl = \'\');',
regexp => qr/^
- \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '');\E
+ \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2440,7 +2440,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub3;',
regexp => qr/^
- \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2449,7 +2449,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub4;',
regexp => qr/^
- \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index e7377d4583..5844f994c5 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -5905,7 +5905,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -5940,6 +5940,15 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_database AS \"%s\"",
+ gettext_noop("Database level DDLs"));
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_table AS \"%s\"",
+ gettext_noop("Table level DDLs"));
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6025,6 +6034,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6041,6 +6051,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6054,6 +6065,11 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ {
+ appendPQExpBufferStr(&buf,
+ ", pubddl_database, pubddl_table");
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6100,6 +6116,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols += 2;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6114,6 +6132,11 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ {
+ printTableAddHeader(&cont, gettext_noop("Database level DDL"), true, align);
+ printTableAddHeader(&cont, gettext_noop("Table level DDL"), true, align);
+ }
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6124,6 +6147,11 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ {
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
+ printTableAddCell(&cont, PQgetvalue(res, i, 10), false, false);
+ }
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 29b1856665..f14074ff76 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -20,6 +20,7 @@
#include "catalog/genbki.h"
#include "catalog/objectaddress.h"
#include "catalog/pg_publication_d.h"
+#include "tcop/utility.h"
/* ----------------
* pg_publication definition. cpp turns this into
@@ -54,6 +55,12 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if database level ddls are published */
+ bool pubddl_database;
+
+ /* true if table level ddls are published */
+ bool pubddl_table;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +79,8 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationActions;
typedef struct PublicationDesc
@@ -157,5 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..073f46ef1a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -15,6 +15,7 @@
#define DEFREM_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_publication.h"
#include "nodes/params.h"
#include "parser/parse_node.h"
#include "tcop/dest.h"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 398c0f38f6..180ebd359e 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | t | t | t | f | f | f | t | t
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.32.0
[application/octet-stream] v4-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch (4.1K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/4-v4-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch)
download | inline diff:
From abe5b204d5473488f4861eed5b6acd783a84123f Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 22 Mar 2022 23:47:52 +0000
Subject: [PATCH 4/6] Enable replication of CREATE MATERIALIZED VIEW AS stmt.
Add tests for both CREATE MATERIALIZED VIEW and CREATE VIEW.
---
src/backend/tcop/utility.c | 42 +++++++++++++++++++++++--
src/test/subscription/t/030_rep_ddls.pl | 18 +++++++++++
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1c8dae4f6b..b764d9df3d 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1140,6 +1140,45 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
}
break;
+ /*
+ * CreateTableAsStmt can create either a table a materialized view
+ * and they are handled differently.
+ */
+ case T_CreateTableAsStmt:
+ {
+ CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+ switch(stmt->objtype)
+ {
+ case OBJECT_TABLE:
+ /*
+ * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
+ * 1. A normal CREATE TABLE string that get's logged and replicated via
+ * DDL replication.
+ * 2. Insertions that get replicated by DML replication.
+ */
+ break;
+ case OBJECT_MATVIEW:
+ /*
+ * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ default:
+ break;
+ }
+ break;
+ }
+
/*
* Secondly, commands that may be allowed in Table level DDL replication.
* These are currently handled in the later execution path of the command.
@@ -1155,7 +1194,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/* DropStmt depends on the removeType */
case T_DropStmt:
{
- DropStmt* stmt = (DropStmt*) parsetree;
+ DropStmt *stmt = (DropStmt *) parsetree;
switch (stmt->removeType)
{
/* Maybe allowed in Table level DDL replication, handled in later code path */
@@ -1206,7 +1245,6 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
case T_RuleStmt:
case T_CreateSeqStmt:
case T_AlterSeqStmt:
- case T_CreateTableAsStmt:
case T_RefreshMatViewStmt:
case T_CreatePLangStmt:
case T_CreateConversionStmt:
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index c88c4ea1c0..562efe2cf7 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -226,6 +226,24 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+# Test CREATE MATERIALIZED VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE MATERIALIZED VIEW s1.matview1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+is($result, qq($result_sub), 'CREATE of s1.matview1 is replicated');
+
+# Test CREATE VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.view1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] v4-0003-Logical-replication-of-DDL-messages.patch (41.2K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/5-v4-0003-Logical-replication-of-DDL-messages.patch)
download | inline diff:
From 4a4d4326a57b37b1b50aee6488a8b1bcdb6aedd1 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 17:17:39 +0000
Subject: [PATCH 3/6] Logical replication of DDL messages
Integration with pgoutput:
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced
for this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
Logical replication worker change:
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to SUBREL_STATE_INIT,
So that DML replication can progress on this new table without
manually running "ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
TAP test:
A new TAP test 030_rep_ddl.pl is added. We mainly focused on
testing the happy path of database level replication so far.
Corner case DDLs and table level DDL replication are still to be
carefully tested.
---
.../test_decoding/expected/ddlmessages.out | 26 +-
contrib/test_decoding/sql/ddlmessages.sql | 5 +-
src/backend/replication/logical/proto.c | 63 ++++-
src/backend/replication/logical/worker.c | 264 ++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 82 +++++-
src/backend/tcop/utility.c | 14 +-
src/include/replication/logicalproto.h | 10 +-
src/test/subscription/t/004_sync.pl | 2 +-
src/test/subscription/t/006_rewrite.pl | 2 +-
src/test/subscription/t/008_diff_schema.pl | 2 +-
src/test/subscription/t/009_matviews.pl | 2 +-
src/test/subscription/t/012_collation.pl | 2 +-
src/test/subscription/t/013_partition.pl | 8 +-
src/test/subscription/t/030_rep_ddls.pl | 237 ++++++++++++++++
14 files changed, 684 insertions(+), 35 deletions(-)
create mode 100644 src/test/subscription/t/030_rep_ddls.pl
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 79284f9def..0376f36c24 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -2,6 +2,9 @@
SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
?column?
----------
@@ -20,23 +23,22 @@ BEGIN;
CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
-SELECT pg_drop_replication_slot('regression_slot');
-DROP TABLE test_ddlmessage;
-DROP publication mypub;
- data
---------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
(1 row)
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index 211497ee22..c23610f9b4 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -3,6 +3,10 @@ SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
+
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
CREATE TABLE test_ddlmessage (id serial unique, data int);
@@ -20,7 +24,6 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e2d2..3cfa94dd8c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -662,6 +662,63 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
pq_sendbytes(out, message, sz);
}
+/*
+ * Read DDL MESSAGE from stream
+ */
+const char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ const char **role,
+ const char **search_path,
+ bool *transactional,
+ Size *sz)
+{
+ uint8 flags;
+ const char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *role = pq_getmsgstring(in);
+ *search_path = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* encode and send message flags */
+ if (transactional)
+ flags |= MESSAGE_TRANSACTIONAL;
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendstring(out, role);
+ pq_sendstring(out, search_path);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
/*
* Write relation description to the output stream.
*/
@@ -1218,6 +1275,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 9181d3e863..f9682dcaac 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -180,6 +181,8 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "tcop/pquery.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -346,6 +349,10 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
+static void apply_execute_sql_command(const char *cmdstr,
+ const char* role,
+ const char* search_path,
+ bool isTopLevel);
/* Compute GID for two_phase transactions */
static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
@@ -2446,6 +2453,259 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/*
+ * Handle generic messages.
+ */
+static void
+apply_handle_ddlmessage(StringInfo s)
+{
+ XLogRecPtr lsn;
+ bool transactional;
+ Size sz;
+ const char *prefix;
+ const char *role;
+ const char *search_path;
+ const char *msg;
+
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+
+ apply_execute_sql_command(msg, role, search_path, true);
+}
+
+/*
+ * Add context to the errors produced by apply_execute_sql_command().
+ */
+static void
+execute_sql_command_error_cb(void *arg)
+{
+ errcontext("during execution of SQL statement: %s", (char *) arg);
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE stmt only */
+ char *relname = NULL; /* For CREATE TABLE stmt only */
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ /*
+ * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, CreateStmt may have been freeed up.
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ RangeVar *rv = cstmt->relation;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+ }
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * OK to analyze, rewrite, and plan this query.
+ *
+ * Switch to appropriate context for constructing query and plan trees
+ * (these can't be in the transaction context, as that will get reset
+ * when the command is COMMIT/ROLLBACK). If we have multiple
+ * parsetrees, we use a separate context for each one, so that we can
+ * free that memory before moving on to the next one. But for the
+ * last (or only) parsetree, just use MessageContext, which will be
+ * reset shortly after completion anyway. In event of an error, the
+ * per_parsetree_context will be deleted when MessageContext is reset.
+ */
+ if (lnext(parsetree_list, parsetree_item) != NULL)
+ {
+ per_parsetree_context =
+ AllocSetContextCreate(MessageContext,
+ "per-parsetree message context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+ }
+ else
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(
+ command,
+ cmdstr,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(
+ querytree_list, cmdstr, 0, NULL);
+
+ /*
+ * Done with the snapshot used for parsing/planning.
+ *
+ * While it looks promising to reuse the same snapshot for query
+ * execution (at least for simple protocol), unfortunately it causes
+ * execution to use a snapshot that has been acquired before locking
+ * any of the tables mentioned in the query. This creates user-
+ * visible anomalies, so refrain. Refer to
+ * https://postgr.es/m/flat/5075D8DF.6050500@fuzzy.cz for details.
+ */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in MessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ cmdstr,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ isTopLevel,
+ true,
+ receiver,
+ receiver,
+ NULL);
+ (*receiver->rDestroy) (receiver);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set
+ * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * new table can be replicated without having to manually run
+ * "alter subscription ... refresh publication"
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+ }
+
+ /*
+ * Restore the GUC variables we set above.
+ */
+ AtEOXact_GUC(true, save_nestlevel);
+
+ /* protect against stack resets during CONCURRENTLY processing */
+ if (error_context_stack == &errcallback)
+ error_context_stack = errcallback.previous;
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
/*
* Logical replication protocol message dispatcher.
@@ -2511,6 +2771,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddlmessage(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index fe5accca57..a8a9703e71 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -255,6 +259,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -271,6 +276,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1650,8 +1656,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1690,6 +1696,70 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char * role,
+ const char *search_path, Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+ ListCell *lc;
+
+ /* Reload publications if needed before use. */
+ if (!publications_valid)
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ if (data->publications)
+ list_free_deep(data->publications);
+
+ data->publications = LoadPublications(data->publication_names);
+ MemoryContextSwitchTo(oldctx);
+ publications_valid = true;
+ }
+
+ /* Check if ddl replication is turned on for the publications */
+ foreach(lc, data->publications)
+ {
+ Publication *pub = (Publication *) lfirst(lc);
+ /* TODO need to check relid for table level DDLs */
+ if (!pub->pubactions.pubddl_database && !pub->pubactions.pubddl_table)
+ return;
+ }
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * Output BEGIN if we haven't yet. Avoid for non-transactional
+ * messages.
+ */
+ if (transactional)
+ {
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+ }
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ transactional,
+ prefix,
+ role,
+ search_path,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1975,7 +2045,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl_database =
+ entry->pubactions.pubddl_table = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2033,6 +2105,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl_database = false;
+ entry->pubactions.pubddl_table = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2146,6 +2220,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl_database |= pub->pubactions.pubddl_database;
+ entry->pubactions.pubddl_table |= pub->pubactions.pubddl_table;
/*
* We want to publish the changes as the top-most ancestor
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f17a8aa75e..1c8dae4f6b 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1268,13 +1268,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
switch (nodeTag(parsetree))
{
/*
@@ -2104,6 +2097,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8ff3..28ff562d62 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,14 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
+extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
+ const char **role, const char **search_path,
+ bool *transactional, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index cf61fc1e0f..698c5114e6 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -33,7 +33,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index c924ff35f7..6c9b055eb0 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -23,7 +23,7 @@ $node_subscriber->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index 67b4026afa..2902e6fc34 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -32,7 +32,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 1ce696d4a4..6c586877a1 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -19,7 +19,7 @@ $node_subscriber->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/012_collation.pl b/src/test/subscription/t/012_collation.pl
index 2182f7948e..e1b5050663 100644
--- a/src/test/subscription/t/012_collation.pl
+++ b/src/test/subscription/t/012_collation.pl
@@ -76,7 +76,7 @@ $node_subscriber->safe_psql('postgres',
# set up publication, subscription
$node_publisher->safe_psql('postgres',
- q{CREATE PUBLICATION pub1 FOR ALL TABLES});
+ q{CREATE PUBLICATION pub1 FOR ALL TABLES WITH (ddl = '')});
$node_subscriber->safe_psql('postgres',
qq{CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = false)}
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 66e63e755e..de967eaaf2 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -25,9 +25,9 @@ $node_subscriber2->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
# publisher
-$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1");
+$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1 WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_all FOR ALL TABLES");
+ "CREATE PUBLICATION pub_all FOR ALL TABLES WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
"CREATE TABLE tab1 (a int PRIMARY KEY, b text) PARTITION BY LIST (a)");
$node_publisher->safe_psql('postgres',
@@ -424,12 +424,12 @@ $node_publisher->safe_psql('postgres',
# and child tables are present but changes will be replicated via the parent's
# identity and only once.
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# for tab4, we publish changes through the "middle" partitioned table
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# prepare data for the initial sync
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
new file mode 100644
index 0000000000..c88c4ea1c0
--- /dev/null
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -0,0 +1,237 @@
+
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $node_subscriber2 = PostgreSQL::Test::Cluster->new('subscriber2');
+$node_subscriber2->init(allows_streaming => 'logical');
+$node_subscriber2->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber2->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+$node_subscriber2->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+# mypub has pubddl_database on
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+# mypub2 has pubddl_database off
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub2 FOR ALL TABLES with (ddl = '');");
+$node_subscriber2->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub2 CONNECTION '$publisher_connstr' PUBLICATION mypub2;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test smae simple CREATE TABLE command is not replicated to subscriber2 (ddl off)
+# Test ALTER TABLE command is replicated on table test_rep
+# Test CREATE INDEX is replicated to subscriber
+# Test CREATE FUNCTION command is replicated to subscriber
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+$node_publisher->safe_psql('postgres', "CREATE INDEX nameindex on test_rep (name)");
+$node_publisher->safe_psql('postgres', qq{CREATE OR REPLACE FUNCTION totalRecords()
+RETURNS integer AS \$total\$
+declare
+ total integer;
+BEGIN
+ SELECT count(*) into total FROM test_rep;
+ RETURN total;
+END;
+\$total\$ LANGUAGE plpgsql;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't1';");
+is($result, qq(0), 'CREATE of t1 is not replicated to subscriber2');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(1), 'CREATE INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT totalRecords();");
+is($result, qq(2), 'CREATE of function totalRecords replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'CREATE FUNCTION totalrecords is not replicated to subscriber2');
+
+# Test ALTER TABLE DROP
+# Test DROP INDEX
+# Test DROP FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->safe_psql('postgres', "DROP INDEX nameindex;");
+$node_publisher->safe_psql('postgres', "DROP FUNCTION totalRecords;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(0), 'DROP INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'DROP FUNCTION totalrecords replicated');
+
+
+# TODO figure out how to set ON_ERROR_STOP = 0 in this test
+# Test failed CREATE/ALTER TABLE on publisher doesn't break replication
+# Table t1 already exits so expect the command to fail
+#$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+#$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (103, 'data103', 1013);");
+
+#$node_publisher->wait_for_catchup('mysub');
+# Verify replication still works
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+#is($result, qq(1), 'DELETE from test_rep replicated');
+
+# Test DDLs inside txn block
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE t2 (a int, b varchar);
+ALTER TABLE test_rep ADD c3 int;
+INSERT INTO test_rep VALUES (3, 'data3', 3);
+COMMIT;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t2;");
+is($result, qq(0), 'CREATE t2 replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep;");
+is($result, qq(2), 'ALTER test_rep ADD replicated');
+
+# Test toggling pubddl_database option off
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = '');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(f|f), 'pubddl_database turned off on mypub');
+$node_publisher->safe_psql('postgres', "CREATE TABLE t3 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't3';");
+is($result, qq(0), 'CREATE t3 is not replicated');
+
+# Test toggling pubddl_database option on
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = 'database');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(t|t), 'pubddl_database turned on on mypub');
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# A somewhat complicated test in plpgsql block with trigger
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE foo (a int);
+CREATE INDEX foo_idx ON foo (a);
+ALTER TABLE foo ADD COLUMN b timestamptz;
+CREATE FUNCTION foo_ts()
+RETURNS trigger AS $$
+BEGIN
+NEW.b := current_timestamp;
+RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
+FOR EACH ROW EXECUTE FUNCTION foo_ts();
+INSERT INTO foo VALUES (1);
+COMMIT;});
+$result = $node_publisher->safe_psql('postgres', "SELECT b from foo where a = 1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result_sub = $node_subscriber->safe_psql('postgres', "SELECT b from foo where a = 1;");
+is($result, qq($result_sub), 'timestamp of insert matches');
+
+# Test CREATE SCHEMA stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE SCHEMA s1");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_namespace WHERE nspname = 's1';");
+is($result, qq(1), 'CREATE SCHEMA s1 is replicated');
+
+# Test CREATE TABLE in new schema s1 followed by insert
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (2, 'b');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t1;");
+is($result, qq(2), 'CREATE TABLE s1.t1 is replicated');
+
+# Test replication works as expected with mismatched search_path on publisher and subscriber
+$node_publisher->append_conf('postgresql.conf', 'search_path = \'s1, public\'');
+$node_publisher->restart;
+# CREATE unqualified table t2, it is s1.t2 under the modified search_path
+$node_publisher->safe_psql('postgres', "CREATE TABLE t2 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (2, 'b');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (3, 'c');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t2;");
+is($result, qq(3), 'CREATE TABLE s1.t2 is replicated');
+
+# Test owner of new table on subscriber matches the owner on publisher
+$node_publisher->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_subscriber->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_publisher->safe_psql('postgres', "SET SESSION AUTHORIZATION 'ddl_replication_user'; CREATE TABLE t5 (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
+is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+
+#TODO TEST certain DDLs are not replicated
+
+pass "DDL replication tests passed!";
+
+$node_subscriber->stop;
+$node_subscriber2->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.32.0
[application/octet-stream] v4-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch (61.3K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/6-v4-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch)
download | inline diff:
From 6653c92ae7a927ab9cc3e5c65e128377a86c6f7a Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:57:23 +0000
Subject: [PATCH 2/6] Support logical logging and decoding of DDL command
string.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current consumers.
The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on
the configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
Support logical decoding of the new WAL record xl_logical_ddl_message.
This is similar to the logical decoding of xl_logical_message. Tests
for this change are added in the test_decoding plugin.
---
contrib/test_decoding/Makefile | 4 +-
.../test_decoding/expected/ddlmessages.out | 42 ++++
contrib/test_decoding/sql/ddlmessages.sql | 28 +++
contrib/test_decoding/test_decoding.c | 61 +++++-
src/backend/access/rmgrdesc/Makefile | 1 +
.../access/rmgrdesc/logicalddlmsgdesc.c | 54 +++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 52 +++++
src/backend/commands/tablecmds.c | 43 +++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 99 +++++++++
src/backend/replication/logical/decode.c | 56 +++++
src/backend/replication/logical/logical.c | 91 ++++++++
.../replication/logical/reorderbuffer.c | 204 +++++++++++++++++-
src/backend/tcop/utility.c | 201 ++++++++++++++++-
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/include/access/rmgrlist.h | 1 +
src/include/commands/tablecmds.h | 3 +-
src/include/replication/ddlmessage.h | 47 ++++
src/include/replication/decode.h | 1 +
src/include/replication/output_plugin.h | 29 +++
src/include/replication/reorderbuffer.h | 39 ++++
src/test/regress/expected/publication.out | 16 +-
23 files changed, 1054 insertions(+), 21 deletions(-)
create mode 100644 contrib/test_decoding/expected/ddlmessages.out
create mode 100644 contrib/test_decoding/sql/ddlmessages.sql
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index b220906479..e58a76d3f1 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -5,7 +5,9 @@ PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- spill slot truncate stream stats twophase twophase_stream
+ spill slot truncate stream stats twophase twophase_stream \
+ ddlmessages
+
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
new file mode 100644
index 0000000000..79284f9def
--- /dev/null
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -0,0 +1,42 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+(6 rows)
+
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
new file mode 100644
index 0000000000..211497ee22
--- /dev/null
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -0,0 +1,28 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index 08d366a594..fab63bc2a8 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -76,6 +76,11 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
TransactionId xid,
const char *gid);
@@ -116,6 +121,11 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
int nrelations, Relation relations[],
@@ -141,6 +151,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
cb->message_cb = pg_decode_message;
+ cb->ddlmessage_cb = pg_decode_ddlmessage;
cb->filter_prepare_cb = pg_decode_filter_prepare;
cb->begin_prepare_cb = pg_decode_begin_prepare_txn;
cb->prepare_cb = pg_decode_prepare_txn;
@@ -153,6 +164,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pg_decode_stream_commit;
cb->stream_change_cb = pg_decode_stream_change;
cb->stream_message_cb = pg_decode_stream_message;
+ cb->stream_ddlmessage_cb = pg_decode_stream_ddlmessage;
cb->stream_truncate_cb = pg_decode_stream_truncate;
}
@@ -747,7 +759,8 @@ pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
appendStringInfo(ctx->out, "message: transactional: %d prefix: %s, sz: %zu content:",
@@ -756,6 +769,19 @@ pg_decode_message(LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+ appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_stream_start(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
@@ -936,7 +962,8 @@ pg_decode_stream_change(LogicalDecodingContext *ctx,
static void
pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
@@ -948,7 +975,35 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
else
{
appendStringInfo(ctx->out, "streaming message: transactional: %d prefix: %s, sz: %zu content:",
- transactional, prefix, sz);
+ transactional, prefix, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ }
+
+ OutputPluginWrite(ctx, true);
+}
+
+/*
+ * In streaming mode, we don't display the contents for transactional messages
+ * as the transaction can abort at a later point in time. We don't want users to
+ * see the message contents until the transaction is committed.
+ */
+static void
+pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char * role, const char * search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+
+ if (transactional)
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
+ transactional, prefix, role, search_path, sz);
+ }
+ else
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
}
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72fd86..b8e29e8df3 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000000..7a352d540a
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *role = xlrec->message + xlrec->prefix_size;
+ char *search_path = xlrec->message + xlrec->prefix_size + xlrec->role_size;
+ char *message = xlrec->message + xlrec->prefix_size + xlrec->role_size + xlrec->search_path_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
+ xlrec->transactional ? "transactional" : "non-transactional",
+ prefix, role, search_path, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index e1d6ebbd3d..195b9e1189 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -29,6 +29,7 @@
#include "miscadmin.h"
#include "replication/decode.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "storage/standby.h"
#include "utils/builtins.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ddf1363477..ade8549f22 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1156,3 +1156,55 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
+
+/*
+ * Checks if DDL on relation (relid) need xlog for logical replication
+ */
+bool
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+{
+ List *allTablePubs = NIL;
+ List *tablePubs = NIL;
+ ListCell *lc;
+
+ /* Only replicate toplevel DDL command */
+ if (!isTopLevel)
+ return false;
+ if (relid == InvalidOid && !forAllTabPubOnly)
+ return false;
+
+ /*
+ * Log the DDL command if
+ * there is any FOR ALL TABLES publication with pubddl_database on
+ * or
+ * this TABLE belongs to any non FOR ALL publications with pubddl_table on
+ */
+ allTablePubs = GetAllTablesPublications();
+ foreach(lc, allTablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_database)
+ return true;
+ }
+
+ /*
+ * If forAllTabPubOnly is true (i.e. database level replication is required for the DDL
+ * to be logged), we can bail now since no publication has been found with pubddl_database on
+ */
+ if (forAllTabPubOnly)
+ return false;
+
+ tablePubs = GetRelationPublications(relid);
+ foreach(lc, tablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_table)
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1d7db41d17..0a6a48b273 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -80,6 +80,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -1333,13 +1334,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(DropStmt *drop)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
+ bool ddlxlog = XLogLogicalInfoActive();
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1438,10 +1440,37 @@ RemoveRelations(DropStmt *drop)
/* Not there? */
if (!OidIsValid(relOid))
{
+ ddlxlog = false;
DropErrorMsgNonExistent(rel, relkind, drop->missing_ok);
continue;
}
+ /*
+ * Only log DROP RELATION cmd for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on or
+ * every relation to be dropped belongs to any non FOR ALL publications with pubddl_table on
+ */
+ if (ddlxlog)
+ {
+ Oid tableOid = InvalidOid;
+
+ if (relkind == RELKIND_RELATION)
+ tableOid = relOid;
+ else if (relkind == RELKIND_INDEX)
+ tableOid = IndexGetRelation(relOid, true);
+ /*
+ * Other relation types require database level ddl replication and are
+ * already logged in LogLogicalDDLCommand() if needed.
+ */
+ else
+ ddlxlog = false;
+
+ /* DROP RELATION or INDEX are allowed in table level DDL replication */
+ if (tableOid != InvalidOid &&
+ !ddl_need_xlog(tableOid, false, isTopLevel))
+ ddlxlog = false;
+ }
+
/*
* Decide if concurrent mode needs to be used here or not. The
* callback retrieved the rel's persistence for us.
@@ -1485,6 +1514,18 @@ RemoveRelations(DropStmt *drop)
add_exact_object_address(&obj, objects);
}
+ /* Log the Drop command for logical replication */
+ if (ddlxlog)
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
performMultipleDeletions(objects, drop->behavior, flags);
free_object_addresses(objects);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fdeb71..f3eeb67312 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000000..f93573079a
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
+ * Transactional messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ * Non-transactional messages are sent to the plugin at the time when the
+ * logical decoding reads them from XLOG. This also means that transactional
+ * messages won't be delivered if the transaction was rolled back but the
+ * non-transactional one will always be delivered.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
+{
+ xl_logical_ddl_message xlrec;
+ const char *role;
+
+ role = GetUserNameFromId(roleoid, false);
+
+ /*
+ * Force xid to be allocated if we're emitting a transactional message.
+ */
+ if (transactional)
+ {
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+ }
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.transactional = transactional;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.role_size = strlen(role) + 1;
+ xlrec.search_path_size = strlen(namespace_search_path) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, role), xlrec.role_size);
+ XLogRegisterData(namespace_search_path, xlrec.search_path_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6303647fe0..c7d6dbefdd 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -603,6 +604,61 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
message->message + message->prefix_size);
}
+/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ Snapshot snapshot;
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (message->transactional &&
+ !SnapBuildProcessChange(builder, xid, buf->origptr))
+ return;
+ else if (!message->transactional &&
+ (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
+ SnapBuildXactNeedsSkip(builder, buf->origptr)))
+ return;
+
+ snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ message->transactional,
+ message->message,
+ /* first part of message is prefix */
+ message->message + message->prefix_size,
+ /* Second part of message is role*/
+ message->message + message->prefix_size + message->role_size,
+ /* Third part of message is search_path */
+ message->message_size,
+ message->message + message->prefix_size +
+ message->role_size + message->search_path_size);
+}
+
/*
* Consolidated commit record handling between the different form of commit
* records.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 788769dd73..6a2da0131d 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +94,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +226,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +243,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +261,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1205,6 +1216,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char *search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
@@ -1510,6 +1559,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char* search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6887dc23f6..774f3bdf37 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,20 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.role != NULL)
+ pfree(change->data.ddlmsg.role);
+ change->data.ddlmsg.role = NULL;
+ if (change->data.ddlmsg.search_path != NULL)
+ pfree(change->data.ddlmsg.search_path);
+ change->data.ddlmsg.search_path = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -866,6 +880,64 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
}
+/*
+ * A transactional DDL message is queued to be processed upon commit and a
+ * non-transactional DDL message gets processed immediately.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
+{
+ if (transactional)
+ {
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ else
+ {
+ ReorderBufferTXN *txn = NULL;
+ volatile Snapshot snapshot_now = snapshot;
+
+ if (xid != InvalidTransactionId)
+ txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+
+ /* setup snapshot to allow catalog access */
+ SetupHistoricSnapshot(snapshot_now, NULL);
+ PG_TRY();
+ {
+ rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+
+ TeardownHistoricSnapshot(false);
+ }
+ PG_CATCH();
+ {
+ TeardownHistoricSnapshot(true);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+}
+
/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
@@ -1957,6 +2029,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message);
}
+/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
@@ -2335,6 +2430,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -3709,6 +3808,53 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the role including the size */
+ memcpy(data, &role_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.role,
+ role_size);
+ data += role_size;
+
+ /* write the search_path including the size */
+ memcpy(data, &search_path_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.search_path,
+ search_path_size);
+ data += search_path_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4023,6 +4169,18 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4283,8 +4441,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4298,6 +4455,49 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+ Size role_size;
+ Size search_path_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read role */
+ memcpy(&role_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.role = MemoryContextAlloc(rb->context,
+ role_size);
+ memcpy(change->data.ddlmsg.role, data, role_size);
+ Assert(change->data.ddlmsg.role[role_size - 1] == '\0');
+ data += role_size;
+
+ /* read search_path */
+ memcpy(&search_path_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.search_path = MemoryContextAlloc(rb->context,
+ search_path_size);
+ memcpy(change->data.ddlmsg.search_path, data, search_path_size);
+ Assert(change->data.ddlmsg.search_path[search_path_size - 1] == '\0');
+ data += search_path_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f364a9b88a..f17a8aa75e 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -62,6 +62,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
#include "storage/fd.h"
@@ -86,7 +87,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -987,7 +988,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel);
}
break;
@@ -1087,6 +1088,154 @@ standard_ProcessUtility(PlannedStmt *pstmt,
CommandCounterIncrement();
}
+/*
+ * Log a DDL command for logical replication
+ * Some DDLs are only replicated in Database Level DDL replication
+ * Some can be replicated in Table Level DDL replication.
+ *
+ * Currently we focus on supporting Database Level DDL replication
+ */
+static void
+LogLogicalDDLCommand(Node *parsetree, const char *queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* Fisrtly, commands that are only supported in Database level DDL replication */
+ case T_CreateSchemaStmt:
+ case T_CreateStmt:
+ case T_CreateForeignTableStmt:
+ case T_AlterDomainStmt:
+ case T_DefineStmt:
+ case T_CompositeTypeStmt:
+ case T_CreateEnumStmt:
+ case T_CreateRangeStmt:
+ case T_AlterEnumStmt:
+ case T_ViewStmt:
+ case T_CreateFunctionStmt:
+ case T_AlterFunctionStmt:
+ case T_CreateTrigStmt:
+ case T_CreateDomainStmt:
+ case T_CreateCastStmt:
+ case T_CreateOpClassStmt:
+ case T_CreateOpFamilyStmt:
+ case T_AlterOpFamilyStmt:
+ case T_AlterOperatorStmt:
+ case T_AlterTypeStmt:
+ case T_GrantStmt:
+ case T_AlterCollationStmt:
+ /*
+ * Log these stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+
+ /*
+ * Secondly, commands that may be allowed in Table level DDL replication.
+ * These are currently handled in the later execution path of the command.
+ * Because we need to get the relation id which readily available in later
+ * code path.
+ */
+ case T_AlterTableStmt:
+ case T_IndexStmt:
+ case T_RenameStmt: /* TODO */
+ case T_AlterOwnerStmt: /* TODO */
+ break;
+
+ /* DropStmt depends on the removeType */
+ case T_DropStmt:
+ {
+ DropStmt* stmt = (DropStmt*) parsetree;
+ switch (stmt->removeType)
+ {
+ /* Maybe allowed in Table level DDL replication, handled in later code path */
+ case OBJECT_INDEX:
+ case OBJECT_TABLE:
+ break;
+ /* Drop of sequence is by logical replication of sequences separately */
+ case OBJECT_SEQUENCE:
+ break;
+ /* Drop of other objects are allowed in Database level DDL replication only */
+ case OBJECT_VIEW:
+ case OBJECT_MATVIEW:
+ case OBJECT_FOREIGN_TABLE:
+ default:
+ /*
+ * Log these DropStmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+ }
+ }
+ /*
+ * Lastly, rule out DDLs we don't replicate yet in DDL replication
+ * Some of these can be supported, we just need to investigate and run tests.
+ */
+ case T_CreateExtensionStmt:
+ case T_AlterExtensionStmt:
+ case T_AlterExtensionContentsStmt:
+ case T_CreateFdwStmt:
+ case T_AlterFdwStmt:
+ case T_CreateForeignServerStmt:
+ case T_AlterForeignServerStmt:
+ case T_CreateUserMappingStmt:
+ case T_AlterUserMappingStmt:
+ case T_DropUserMappingStmt:
+ case T_ImportForeignSchemaStmt:
+ case T_RuleStmt:
+ case T_CreateSeqStmt:
+ case T_AlterSeqStmt:
+ case T_CreateTableAsStmt:
+ case T_RefreshMatViewStmt:
+ case T_CreatePLangStmt:
+ case T_CreateConversionStmt:
+ case T_CreateTransformStmt:
+ case T_AlterTSDictionaryStmt:
+ case T_AlterTSConfigurationStmt:
+ case T_AlterTableMoveAllStmt:
+ case T_AlterObjectDependsStmt:
+ case T_AlterObjectSchemaStmt:
+ case T_CommentStmt:
+ case T_DropOwnedStmt:
+ case T_AlterDefaultPrivilegesStmt:
+ case T_CreatePolicyStmt:
+ case T_AlterPolicyStmt:
+ case T_SecLabelStmt:
+ case T_CreateAmStmt:
+ case T_CreatePublicationStmt:
+ case T_AlterPublicationStmt:
+ case T_CreateSubscriptionStmt:
+ case T_AlterSubscriptionStmt:
+ case T_DropSubscriptionStmt:
+ case T_CreateStatsStmt:
+ case T_AlterStatsStmt:
+ break;
+ default:
+ break;
+ }
+}
+
/*
* The "Slow" variant of ProcessUtility should only receive statements
* supported by the event triggers facility. Therefore, we always
@@ -1119,6 +1268,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
switch (nodeTag(parsetree))
{
/*
@@ -1321,6 +1477,23 @@ ProcessUtilitySlow(ParseState *pstate,
EventTriggerAlterTableStart(parsetree);
EventTriggerAlterTableRelid(relid);
+ /*
+ * Log the ALTER TABLE command if
+ * There is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
/* ... and do it */
AlterTable(atstmt, lockmode, &atcontext);
@@ -1539,6 +1712,24 @@ ProcessUtilitySlow(ParseState *pstate,
/* ... and do it */
EventTriggerAlterTableStart(parsetree);
+
+ /*
+ * Log CREATE INDEX cmd for logical replication if
+ * there is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on.
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
address =
DefineIndex(relid, /* OID of heap relation */
stmt,
@@ -1755,7 +1946,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt((DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -1976,7 +2167,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
{
switch (stmt->removeType)
{
@@ -1991,7 +2182,7 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(stmt);
+ RemoveRelations(pstate, stmt, isTopLevel);
break;
default:
RemoveObjects(stmt);
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17bb4c..792f438959 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721c97..9de3b8f2eb 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 5d4037f26e..68781365de 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -18,6 +18,7 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "storage/lock.h"
#include "utils/relcache.h"
@@ -27,7 +28,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(DropStmt *drop);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000000..1e8ef22296
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ bool transactional; /* is message transactional? */
+ Size prefix_size; /* length of prefix */
+ Size role_size; /* length of the role that executes the DDL command */
+ Size search_path_size; /* length of the search path */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ * and null-terminated role of length role_size
+ * and null-terminated search_path of length search_path_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
+ size_t size, bool transactional);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index a33c2a718a..697867a6ff 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e697..5b1c245b72 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -88,6 +88,19 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
Size message_size,
const char *message);
+/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Filter changes by origin.
*/
@@ -199,6 +212,20 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
Size message_size,
const char *message);
+/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Callback for streaming truncates from in-progress transactions.
*/
@@ -219,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index f12e75d69b..7fbb473da2 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ char *role;
+ char *search_path;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +441,17 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +518,18 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size message_size, const char *message);
void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 180ebd359e..81cdee0527 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | t | f | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
--
2.32.0
[application/octet-stream] v4-0006-Add-couple-tests-for-DDL-replication-case.patch (4.5K, ../../CAAD30UKJJXSMCb0Xk9ZtS1si4KzbasmYzbJjD8rZxTX2HWnGZA@mail.gmail.com/7-v4-0006-Add-couple-tests-for-DDL-replication-case.patch)
download | inline diff:
From 3fe6b062bc467a4812d92c76be59f91ec44fdb2b Mon Sep 17 00:00:00 2001
From: Borui Yang <boruiyan@amazon.com>
Date: Mon, 11 Apr 2022 18:02:21 +0000
Subject: [PATCH 6/6] Add couple tests for DDL replication case
---
src/test/subscription/t/030_rep_ddls.pl | 80 +++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 3b15c6d9f0..4e05212068 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -268,6 +268,86 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+# TEST Create DomainStmt
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(t), 'CreateDomain Stmt is replicatted');
+
+# TEST AlterDomainStmt
+$node_publisher->safe_psql('postgres', "Alter domain s1.space_check drop not null;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(f), 'ALTER DOMAIN Stmt is replicated');
+
+#TEST DEFINE Stmt
+$node_publisher->safe_psql('postgres', "CREATE AGGREGATE s1.inc_sum(int) (sfunc = int4pl,stype = int,initcond = 10);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='inc_sum';");
+is($result, qq(1), 'Define stmt is replicated');
+
+#TEST CompositeTypeStmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.compfoo AS (f1 int, f2 text);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='compfoo';");
+is($result, qq(1), 'CompositeType Stmt is replicated');
+
+#TEST CreateEnum Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.mood AS ENUM ('sad', 'ok', 'happy');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='mood';");
+is($result, qq(1), 'CreateEnumType Stmt is replicated');
+
+#TEST AlterEnum Stmt
+$node_publisher->safe_psql('postgres', "ALTER TYPE s1.mood RENAME VALUE 'sad' to 'fine';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_enum e, pg_catalog.pg_type t WHERE e.enumtypid = t.oid AND t.typname='mood' AND e.enumlabel='fine';");
+is($result, qq(1), 'AlterEnumType Stmt is replicated');
+
+#TEST CreateRange Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE floatrange AS RANGE (subtype = float8,subtype_diff = float8mi);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='floatrange';");
+is($result, qq(1), 'CreateRange Stmt is replicated');
+
+#TEST VIEW Stmt
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.vista AS SELECT 'Hello World';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_class c where c.relname='vista';");
+is($result, qq(1), 'VIEW Stmt is replicated');
+
+#TEST CreateFunction Stmt
+$node_publisher->safe_psql('postgres', "CREATE FUNCTION s1.add(a integer, b integer) RETURNS integer LANGUAGE SQL IMMUTABLE RETURN a + b;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='add';");
+is($result, qq(1), 'CreateFunction Stmt is replicated');
+
+#TEST CreateCast Stmt
+$node_publisher->safe_psql('postgres', "CREATE CAST (int AS int4) WITH FUNCTION add(int,int) AS ASSIGNMENT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
+is($result, qq(1), 'CreateCast Stmt is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-14 09:26 Dilip Kumar <dilipbalaut@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Dilip Kumar @ 2022-04-14 09:26 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com
On Fri, Apr 8, 2022 at 4:30 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Mar 29, 2022 at 9:47 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > The idea is to force skipping any direct data population (which can
> > > potentially cause data inconsistency on the subscriber)
> > > in CREATE AS and SELECT INTO command on the subscriber by forcing the
> > > skipData flag in the intoClause of the parsetree after
> > > the logical replication worker parses the command. The data sync will
> > > be taken care of by the DML replication after the DDL replication
> > > finishes.
> >
> > Okay, something like that should work, I am not sure it is the best
> > design though.
> >
>
> Even if this works, how will we make Alter Table statement work where
> it needs to rewrite the table? There also I think we can face a
> similar problem if we directly send the statement, once the table will
> be updated due to the DDL statement and then again due to table
> rewrite as that will have a separate WAL.
I agree. But here the bigger question is what is the correct behavior
in case of the Alter Table? I mean for example in the publisher the
table gets rewritten due to the Access Method change then what should
be the behavior of the subscriber. One expected behavior is that on
subscriber also the access method gets changed and the data remains
the same as on the subscriber table(table can be locally rewritten
based on new AM). Which seems quite sensible behavior to me. But if
we want this behavior then we can not replay the logical messages
generated by DML WAL because of table rewrite, otherwise we will get
duplicate data, unless we plan to get rid of the current data and just
get all new data from the publisher. And if we do that then the data
will be as per the latest data in the table based on the publisher, so
I think first we need to define the correct behavior and then we can
design it accordingly.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-14 14:15 Euler Taveira <euler@eulerto.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
0 siblings, 3 replies; 449+ messages in thread
From: Euler Taveira @ 2022-04-14 14:15 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
On Thu, Apr 14, 2022, at 6:26 AM, Dilip Kumar wrote:
> I agree. But here the bigger question is what is the correct behavior
> in case of the Alter Table? I mean for example in the publisher the
> table gets rewritten due to the Access Method change then what should
> be the behavior of the subscriber. One expected behavior is that on
> subscriber also the access method gets changed and the data remains
> the same as on the subscriber table(table can be locally rewritten
> based on new AM). Which seems quite sensible behavior to me. But if
> we want this behavior then we can not replay the logical messages
> generated by DML WAL because of table rewrite, otherwise we will get
> duplicate data, unless we plan to get rid of the current data and just
> get all new data from the publisher. And if we do that then the data
> will be as per the latest data in the table based on the publisher, so
> I think first we need to define the correct behavior and then we can
> design it accordingly.
You should forbid it. Unless you can decompose the command into multiple SQL
commands to make it a safe operation for logical replication.
Let's say you want to add a column with a volatile default.
ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random();
If you replicate the DDL command as is, you will have different data
downstream. You should forbid it. However, this operation can be supported if
the DDL command is decomposed in multiple steps.
-- add a new column without DEFAULT to avoid rewrite
ALTER TABLE foo ADD COLUMN bar double precision;
-- future rows could use the DEFAULT expression
-- it also doesn't rewrite the table
ALTER TABLE foo ALTER COLUMN bar SET DEFAULT random();
-- it effectively rewrites the table
-- all rows are built from one source node
-- data will be the same on all nodes
UPDATE foo SET bar = random();
The ALTER TABLE ... ALTER COLUMN ... TYPE has a similar issue. This DDL command
can be decomposed to avoid the rewrite. If you are changing the data type, in
general, you add a new column and updates all rows doing the proper conversion.
(If you are updating in batches, you usually add a trigger to automatically
adjust the new column value for INSERTs and UPDATEs. Another case is when you
are reducing the the typmod (for example, varchar(100) to varchar(20)). In this
case, the DDL command can de decomposed removing the typmod information (ALTER
TABLE ... ALTER COLUMN ... TYPE varchar) and replacing it with a CHECK
constraint.
I didn't review this patch in depth but we certainly need to impose some DDL
restrictions if we are replicating DDLs. There are other cases that should be
treated accordingly such as a TABLESPACE specification or a custom data type.
--
Euler Taveira
EDB https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-14 20:31 Zheng Li <zhengli10@gmail.com>
parent: Euler Taveira <euler@eulerto.com>
2 siblings, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-14 20:31 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
> You should forbid it. Unless you can decompose the command into multiple SQL
> commands to make it a safe operation for logical replication.
>
> Let's say you want to add a column with a volatile default.
>
> ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random();
>
> If you replicate the DDL command as is, you will have different data
> downstream. You should forbid it. However, this operation can be supported if
> the DDL command is decomposed in multiple steps.
>
> -- add a new column without DEFAULT to avoid rewrite
> ALTER TABLE foo ADD COLUMN bar double precision;
>
> -- future rows could use the DEFAULT expression
> -- it also doesn't rewrite the table
> ALTER TABLE foo ALTER COLUMN bar SET DEFAULT random();
>
> -- it effectively rewrites the table
> -- all rows are built from one source node
> -- data will be the same on all nodes
> UPDATE foo SET bar = random();
>
> The ALTER TABLE ... ALTER COLUMN ... TYPE has a similar issue. This DDL command
> can be decomposed to avoid the rewrite. If you are changing the data type, in
> general, you add a new column and updates all rows doing the proper conversion.
> (If you are updating in batches, you usually add a trigger to automatically
> adjust the new column value for INSERTs and UPDATEs. Another case is when you
> are reducing the the typmod (for example, varchar(100) to varchar(20)). In this
> case, the DDL command can be decomposed removing the typmod information (ALTER
> TABLE ... ALTER COLUMN ... TYPE varchar) and replacing it with a CHECK
> constraint.
>
> I didn't review this patch in depth but we certainly need to impose some DDL
> restrictions if we are replicating DDLs. There are other cases that should be
> treated accordingly such as a TABLESPACE specification or a custom data type.
This is helpful. Thanks.
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-15 02:08 Zheng Li <zhengli10@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-15 02:08 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; Ajin Cherian <itsajin@gmail.com>; +Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
> > But then this could be true for DML as well right? Like after
> > replicating the function to the subscriber if we are sending the DML
> > done by function then what's the problem in DDL. I mean if there is
> > no design issue in implementing this then I don't think there is much
> > point in blocking the same or even providing configuration for this.
>
> Agreed. I'll unblock DDLs in functions/procedures and test it out. I
> find out some DDLs in functions are replicated multiple times on the
> subscriber while they should only be replicated once. Still trying to
> figure out why.
Here is the patch unblocking DDLs in function and procedures. Also
fixed a bug where DDL with sub-command may get logged twice.
github commit of the same patch:
https://github.com/zli236/postgres/commit/d0fe6065ee3cb6051e5b026f17b82f5220903e6f
Regards,
Zheng
Attachments:
[application/octet-stream] v4-0007-Enable-logging-and-replication-of-DDLs-in-function.patch (17.5K, ../../CAAD30UKRGo_VrdFKYOzWLLRnynjU7DC0uw9mb8LWCPFSnOxXAg@mail.gmail.com/2-v4-0007-Enable-logging-and-replication-of-DDLs-in-function.patch)
download | inline diff:
From d0fe6065ee3cb6051e5b026f17b82f5220903e6f Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 15 Apr 2022 02:02:10 +0000
Subject: [PATCH 7/7] Enable logging and replication of DDLs executed inside
function and procedures. Also fixed a bug where DDL with SUBCOMMAND gets
logged twice.
---
.../test_decoding/expected/ddlmessages.out | 47 +++++++++++++++++--
contrib/test_decoding/sql/ddlmessages.sql | 19 +++++++-
src/backend/catalog/pg_publication.c | 5 +-
src/backend/commands/tablecmds.c | 6 +--
src/backend/tcop/utility.c | 26 +++++-----
src/include/catalog/pg_publication.h | 2 +-
src/include/commands/tablecmds.h | 3 +-
src/test/subscription/t/030_rep_ddls.pl | 38 +++++++++++++++
8 files changed, 121 insertions(+), 25 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 0376f36c24..823029d03d 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -11,7 +11,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d
init
(1 row)
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -24,9 +24,9 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
@@ -34,6 +34,44 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+SELECT func_ddl ('tab_from_func');
+ func_ddl
+----------
+
+(1 row)
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
+ LANGUAGE plpgsql;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ BEGIN
+ table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
+ table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
+ COMMIT
+(7 rows)
+
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
@@ -41,4 +79,5 @@ SELECT pg_drop_replication_slot('regression_slot');
(1 row)
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index c23610f9b4..3082671488 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -9,7 +9,7 @@ SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -25,7 +25,24 @@ ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+
+SELECT func_ddl ('tab_from_func');
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ade8549f22..8d6127fb46 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1161,15 +1161,12 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
* Checks if DDL on relation (relid) need xlog for logical replication
*/
bool
-ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly)
{
List *allTablePubs = NIL;
List *tablePubs = NIL;
ListCell *lc;
- /* Only replicate toplevel DDL command */
- if (!isTopLevel)
- return false;
if (relid == InvalidOid && !forAllTabPubOnly)
return false;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0a6a48b273..2d1a8864ea 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1334,14 +1334,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
- bool ddlxlog = XLogLogicalInfoActive();
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1467,7 +1467,7 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
/* DROP RELATION or INDEX are allowed in table level DDL replication */
if (tableOid != InvalidOid &&
- !ddl_need_xlog(tableOid, false, isTopLevel))
+ !ddl_need_xlog(tableOid, false))
ddlxlog = false;
}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 68d45f3101..a64301b1f8 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -87,7 +87,8 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -988,7 +989,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(pstate, stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1128,7 +1129,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1170,7 +1171,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1221,7 +1222,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1318,7 +1319,7 @@ ProcessUtilitySlow(ParseState *pstate,
* Consider logging the DDL command if logical logging is enabled and this is
* a complete top level query.
*/
- if (XLogLogicalInfoActive() && isTopLevel)
+ if (XLogLogicalInfoActive())
LogLogicalDDLCommand(parsetree, queryString);
}
@@ -1530,7 +1531,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1766,7 +1768,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on.
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1993,7 +1996,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel, isCompleteQuery);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -2214,7 +2217,8 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery)
{
switch (stmt->removeType)
{
@@ -2229,7 +2233,7 @@ ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(pstate, stmt, isTopLevel);
+ RemoveRelations(pstate, stmt, isCompleteQuery);
break;
default:
RemoveObjects(stmt);
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index f14074ff76..c3f2743ed2 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -166,6 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
-extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 68781365de..24106de2e5 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -28,7 +28,8 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop,
+ bool isCompleteQuery);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 4e05212068..34b9d51eb1 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -348,6 +348,44 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST DDL in function
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+\$\$
+LANGUAGE plpgsql;});
+
+$node_publisher->safe_psql('postgres', "SELECT func_ddl('func_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.func_table where c3 = 22;");
+is($result, qq(1), 'DDLs in function are replicated');
+
+#TEST DDL in procedure
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE procedure proc_ddl (tname varchar(20))
+LANGUAGE plpgsql AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END \$\$;});
+
+$node_publisher->safe_psql('postgres', "CALL proc_ddl('proc_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
+is($result, qq(1), 'DDLs in procedure are replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-04-29 17:34 Zheng Li <zhengli10@gmail.com>
parent: Euler Taveira <euler@eulerto.com>
2 siblings, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-04-29 17:34 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
Hello,
> You should forbid it. Unless you can decompose the command into multiple SQL
> commands to make it a safe operation for logical replication.
>
> Let's say you want to add a column with a volatile default.
>
> ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random();
>
> If you replicate the DDL command as is, you will have different data
> downstream. You should forbid it. However, this operation can be supported if
> the DDL command is decomposed in multiple steps.
>
> -- add a new column without DEFAULT to avoid rewrite
> ALTER TABLE foo ADD COLUMN bar double precision;
>
> -- future rows could use the DEFAULT expression
> -- it also doesn't rewrite the table
> ALTER TABLE foo ALTER COLUMN bar SET DEFAULT random();
>
> -- it effectively rewrites the table
> -- all rows are built from one source node
> -- data will be the same on all nodes
> UPDATE foo SET bar = random();
I looked more into this. In order to support statements like "ALTER
TABLE foo ADD COLUMN bar double precision DEFAULT random();", we have
two potential solutions, but both of them are non-trivial to
implement:
1. As Euler pointed out, we could decompose the statement on the
publisher into multiple statements so that the table rewrite (using
volatile function) is handled by a DML sub-command. The decomposition
requires changes in parse analysis/transform. We also need new logic
to assemble the decomposed DDL commands string back from the parse
trees so we can log them for logical replication.
2. Force skipping table rewrite when executing the same command on the
subscriber, and let DML replication replicate the table rewrite from
the publisher. The problem is table rewrite is not replicated at all
today, and it doesn't seem easy to just enable it for logical
replication. Table rewrite is an expensive operation involving heap
file swap, details can be found in ATRewriteTables().
In light of this, I propose to temporarily block replication of such
DDL command on the replication worker until we figure out a better
solution. This is implemented in patch
0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch.
Notice only DDL statements that rewrite table using a VOLATILE
expression will be blocked. I don't see a problem replicating
non-volatile expression.
Here is the github commit of the same patch:
https://github.com/zli236/postgres/commit/1e6115cb99a1286a61cb0a6a088f7476da29d0b9
> The ALTER TABLE ... ALTER COLUMN ... TYPE has a similar issue. This DDL command
> can be decomposed to avoid the rewrite. If you are changing the data type, in
> general, you add a new column and updates all rows doing the proper conversion.
> (If you are updating in batches, you usually add a trigger to automatically
> adjust the new column value for INSERTs and UPDATEs. Another case is when you
> are reducing the the typmod (for example, varchar(100) to varchar(20)). In this
> case, the DDL command can de decomposed removing the typmod information (ALTER
> TABLE ... ALTER COLUMN ... TYPE varchar) and replacing it with a CHECK
> constraint.
I tested ALTER TABLE ... ALTER COLUMN ... TYPE. It seems to be working
fine. Is there a particular case you're concerned about?
> --
> Euler Taveira
> EDB https://www.enterprisedb.com/
>
Regards,
Zheng Li
Attachments:
[application/octet-stream] 0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch (4.8K, ../../CAAD30ULUBpNegQGEx4_CO-eoz4Z1eYmfOKOTPC7ajmT5_n1mVQ@mail.gmail.com/2-0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch)
download | inline diff:
From 1e6115cb99a1286a61cb0a6a088f7476da29d0b9 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 29 Apr 2022 16:57:03 +0000
Subject: [PATCH 10/10] Fail replication worker on DDL command that rewrites
table using volatile functions, such as ALTER TABLE tab ADD COLUMN col
DEFAULT volatile_expr. This is to avoid data mismatch compared to the
publisher. We can potentially unblock this type of command when table rewrite
is supported in logical replication.
---
src/backend/replication/logical/worker.c | 44 ++++++++++++++++++++++++
src/test/subscription/t/030_rep_ddls.pl | 22 ++++++++++++
2 files changed, 66 insertions(+)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index f5274531b7..93281cb88d 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -157,6 +157,7 @@
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
+#include "parser/parse_expr.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -2548,6 +2549,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
commandTag = CreateCommandTag((Node *)command);
+ /* The following DDL commands need special handling */
+
/*
* Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
@@ -2595,6 +2598,47 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
sstmt->intoClause->skipData = true;
}
}
+ /*
+ * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
+ * Until we support logical replication of table rewrite, see ATRewriteTables()
+ * for details on table rewrite.
+ */
+ else if (commandTag == CMDTAG_ALTER_TABLE)
+ {
+ AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
+ ListCell *lc;
+
+ foreach(lc, atstmt->cmds)
+ {
+ AlterTableCmd *cmd = lfirst_node(AlterTableCmd, lc);
+
+ if (cmd->subtype == AT_AddColumn)
+ {
+ ColumnDef *colDef;
+ ListCell *c;
+
+ colDef = castNode(ColumnDef, cmd->def);
+ foreach(c, colDef->constraints)
+ {
+ Constraint *con = lfirst_node(Constraint, c);
+
+ if (con->contype == CONSTR_DEFAULT)
+ {
+ Node *expr;
+ ParseState *pstate = make_parsestate(NULL);
+
+ expr = transformExpr(pstate, copyObject(con->raw_expr), EXPR_KIND_COLUMN_DEFAULT);
+ if (contain_volatile_functions(expr))
+ {
+ elog(ERROR,
+ "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
+ cmdstr);
+ }
+ }
+ }
+ }
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 34b9d51eb1..b4df1bfefd 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -386,7 +386,29 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
is($result, qq(1), 'DDLs in procedure are replicated');
+# Test Alter table alter column type stmt is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN name type TEXT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT data_type FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'name';");
+is($result, qq(text), 'Alter table column type stmt is replicated');
+
+# Test Alter table add column default 0.01 is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN non_volatile double precision DEFAULT 0.01;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
+is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+
#TODO TEST certain DDLs are not replicated
+# Test DDL statement that rewrites table with volatile functions are not replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'volatile';");
+is($result, qq(1), 'Alter table add column default random() is executed on the publisher DB.');
+
+$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
pass "DDL replication tests passed!";
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-03 21:30 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-05-03 21:30 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> > >Another somewhat unrelated problem I see with this work is how to save
> > >recursion of the same command between nodes (when the involved nodes
> > >replicate DDLs). For DMLs, we can avoid that via replication origins
> > >as is being done in the patch proposed [1] but not sure how will we
> > >deal with that here?
> >
> > I'll need to investigate "recursion of the same command between
> > nodes", could you provide an example?
> >
>
> See email [1]. I think the same solution should work for your proposal
> as well because I see that LogLogicalMessage includes origin but it is
> better if you can confirm it.
>
> [1] - https://www.postgresql.org/message-id/flat/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail....
Yes, I applied the patches from [1] and can confirm that the solution
for "infinite recursion of the same command(DDL command in this case)
between nodes" works with my patches.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-06 13:24 Amit Kapila <amit.kapila16@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
2 siblings, 2 replies; 449+ messages in thread
From: Amit Kapila @ 2022-05-06 13:24 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com, Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Fri, Apr 8, 2022 at 5:04 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2022-Apr-08, Amit Kapila wrote:
>
> > On Thu, Mar 17, 2022 at 3:36 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> > For runtime conditions, one of the things you have mentioned in that
> > thread is to add schema name in the statement at the required places
> > which this patch deals with in a different way by explicitly sending
> > it along with the DDL statement.
>
> Hmm, ok. The point of the JSON-blob route is that the publisher sends a
> command representation that can be parsed/processed/transformed
> arbitrarily by the subscriber using generic rules; it should be trivial
> to use a JSON tool to change schema A to schema B in any arbitrary DDL
> command, and produce another working DDL command without having to know
> how to write that command specifically.
>
Attached is a set of two patches as an attempt to evaluate this approach.
The first patch provides functions to deparse DDL commands. Currently,
it is restricted to just a simple CREATE TABLE statement, the required
code is extracted from one of the patches posted in the thread [1].
The second patch allows replicating simple CREATE TABLE DDL
replication. To do that we used an event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The
event trigger is automatically removed at the time of drop
publication. The WALSender decodes the WAL and sends it downstream
similar to other DML commands. The subscriber then converts JSON back
to the DDL command string and executes it. In the subscriber, we also
add the newly added rel to pg_subscription_rel so that the DML changes
on the new table can be replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION". Some of the code related
to WAL logging and subscriber-side work is taken from the patch posted
by Zheng in this thread but there are quite a few changes in that as
we don't need schema, role, transaction vs. non-transactional
handling.
Note that for now, we have hacked Create Publication code such that
when the user specifies the "FOR ALL TABLES" clause, we invoke this
new functionality. So, this will work only for "FOR ALL TABLES"
publications. For example, we need to below to replicate the simple
Create Table command.
Publisher:
Create Publication pub1 For All Tables;
Subscriber:
Create Subscription sub1 Connection '...' Publication pub1;
Publisher:
Create Table t1(c1 int);
Subscriber:
\d should show t1.
As we have hacked CreatePublication function for this POC, the
regression tests are not passing but we can easily change it so that
we invoke new functionality with the syntax proposed in this thread or
with some other syntax and we shall do that in the next patch unless
this approach is not worth pursuing.
This POC is prepared by Ajin Cherian, Hou-San, and me.
Thoughts?
[1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
--
With Regards,
Amit Kapila.
Attachments:
[application/octet-stream] v1-0001-Functions-to-deparse-DDL-commands.patch (62.6K, ../../CAA4eK1+6A72+3HNrbg12wGaAy=rYPDS+AFkbePtSj=h++1ir8Q@mail.gmail.com/2-v1-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 1b35ad4b895cad90218812b17e3da4ecfef65a51 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Thu, 5 May 2022 16:57:41 +0530
Subject: [PATCH v1 1/2] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE specific functions. We can extend
it as we need more functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 1487 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +
src/include/tcop/ddl_deparse.h | 9 +
5 files changed, 2253 insertions(+)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..bbd21f3
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,1487 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6d378ff..ba908b3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11881,5 +11881,11 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..e979ee5
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,9 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+
+#endif /* DDL_DEPARSE_H */
--
1.8.3.1
[application/octet-stream] v1-0002-Support-DDL-replication.patch (47.8K, ../../CAA4eK1+6A72+3HNrbg12wGaAy=rYPDS+AFkbePtSj=h++1ir8Q@mail.gmail.com/3-v1-0002-Support-DDL-replication.patch)
download | inline diff:
From 5fa0808aa5ff9f0dd493fef1552e21c5cc7c7b15 Mon Sep 17 00:00:00 2001
From: Amit Kapila <akapila@postgresql.org>
Date: Thu, 5 May 2022 19:04:33 +0530
Subject: [PATCH v1 2/2] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to a simple CREATE TABLE command.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 51 ++++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/commands/event_trigger.c | 44 +++++
src/backend/commands/publicationcmds.c | 42 +++++
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 83 +++++++++
src/backend/replication/logical/decode.c | 40 ++++
src/backend/replication/logical/logical.c | 88 +++++++++
src/backend/replication/logical/proto.c | 52 +++++-
src/backend/replication/logical/reorderbuffer.c | 123 ++++++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 39 +++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 3 +
src/include/replication/ddlmessage.h | 41 +++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 23 +++
src/include/replication/reorderbuffer.h | 30 +++
22 files changed, 899 insertions(+), 7 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..4dc7591
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,51 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index e1d6ebb..b97bbe9 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..8062415 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -2180,3 +2183,44 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+ static int i;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Fixme: This is just for the development purpose and needs to be
+ * removed later.
+ */
+ elog(LOG, "DDL command no. %d", ++i);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ /* We must get the valid command string. */
+ Assert(json_string != NULL);
+
+ LogLogicalDDLMessage("deparse", json_string, strlen(json_string) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 6df0e66..2c88a9f 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -755,6 +757,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -881,6 +884,45 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (stmt->for_all_tables)
+ {
+ CreateEventTrigStmt *ddl_trigg;
+ Node *args = NULL;
+ List *tags = NIL;
+ Oid event_trig_id;
+ char trigger_name[NAMEDATALEN];
+
+ ddl_trigg = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), "pg_deparse_trig_%u",
+ puboid);
+ ddl_trigg->trigname = pstrdup(trigger_name);
+ ddl_trigg->eventname = "ddl_command_end";
+ ddl_trigg->funcname = SystemFuncName("publication_ddl_deparse");
+
+ args = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ tags = list_make1(args);
+ ddl_trigg->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+ event_trig_id = CreateEventTrigger(ddl_trigg);
+
+ /*
+ * Register the event trigger as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, event_trig_id);
+ recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..8e0be0f
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,83 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6303647..14dc9c4 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,45 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 788769d..d01c143 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,9 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +93,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +225,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +242,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +260,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1206,6 +1216,43 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1511,6 +1558,47 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6887dc2..1c04296 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +875,33 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1993,25 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2389,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -3711,6 +3769,36 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4025,6 +4113,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4283,8 +4380,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4300,6 +4396,29 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 7da7823..fc5e42cf 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b197bfd..5ec9c3e 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,9 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -255,6 +258,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -271,6 +275,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1649,8 +1654,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1689,6 +1694,36 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 01d4c22..9649c5b 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ba908b3..7f0d2ca 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11887,5 +11887,8 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
]
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..bc95f3c
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index a33c2a7..697867a 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..fa7a8b8 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,16 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +210,17 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +240,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +259,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index f12e75d..a6a9133 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,14 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +439,14 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +513,15 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +567,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +587,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +663,8 @@ void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size, const char *message);
void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-06 16:51 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-05-06 16:51 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
> Attached is a set of two patches as an attempt to evaluate this approach.
>
> The first patch provides functions to deparse DDL commands. Currently,
> it is restricted to just a simple CREATE TABLE statement, the required
> code is extracted from one of the patches posted in the thread [1].
>
> The second patch allows replicating simple CREATE TABLE DDL
> replication. To do that we used an event trigger and DDL deparsing
> facilities. While creating a publication, we register a command end
> trigger that deparses the DDL as a JSON blob, and WAL logs it. The
> event trigger is automatically removed at the time of drop
> publication. The WALSender decodes the WAL and sends it downstream
> similar to other DML commands. The subscriber then converts JSON back
> to the DDL command string and executes it. In the subscriber, we also
> add the newly added rel to pg_subscription_rel so that the DML changes
> on the new table can be replicated without having to manually run
> "ALTER SUBSCRIPTION ... REFRESH PUBLICATION". Some of the code related
> to WAL logging and subscriber-side work is taken from the patch posted
> by Zheng in this thread but there are quite a few changes in that as
> we don't need schema, role, transaction vs. non-transactional
> handling.
>
> Note that for now, we have hacked Create Publication code such that
> when the user specifies the "FOR ALL TABLES" clause, we invoke this
> new functionality. So, this will work only for "FOR ALL TABLES"
> publications. For example, we need to below to replicate the simple
> Create Table command.
>
> Publisher:
> Create Publication pub1 For All Tables;
>
> Subscriber:
> Create Subscription sub1 Connection '...' Publication pub1;
>
> Publisher:
> Create Table t1(c1 int);
>
> Subscriber:
> \d should show t1.
>
> As we have hacked CreatePublication function for this POC, the
> regression tests are not passing but we can easily change it so that
> we invoke new functionality with the syntax proposed in this thread or
> with some other syntax and we shall do that in the next patch unless
> this approach is not worth pursuing.
>
> This POC is prepared by Ajin Cherian, Hou-San, and me.
>
> Thoughts?
>
> [1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
Hi,
Thanks for exploring this direction.
I read the deparsing thread and your patch. Here is my thought:
1. The main concern on maintainability of the deparsing code still
applies if we want to adapt it for DDL replication.
2. The search_path and role still need to be handled, in the deparsing
code. And I think it's actually more overhead to qualify every object
compared to just logging the search_path and enforcing it on the apply
worker.
3. I'm trying to understand if deparsing helps with edge cases like
"ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random();". I
don't think it helps out of the box. The crux of separating table
rewrite and DDL still needs to be solved for such cases.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-07 04:08 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-05-07 04:08 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Fri, May 6, 2022 at 10:21 PM Zheng Li <zhengli10@gmail.com> wrote:
>
> > Attached is a set of two patches as an attempt to evaluate this approach.
> >
>
> Thanks for exploring this direction.
>
> I read the deparsing thread and your patch. Here is my thought:
> 1. The main concern on maintainability of the deparsing code still
> applies if we want to adapt it for DDL replication.
>
I agree that it adds to our maintainability effort, like every time we
enhance any DDL or add a new DDL that needs to be replicated, we
probably need to change the deparsing code. But OTOH this approach
seems to provide better flexibility. So, in the long run, maybe the
effort is worthwhile. I am not completely sure at this stage which
approach is better but I thought it is worth evaluating this approach
as Alvaro and Robert seem to prefer this idea.
> 2. The search_path and role still need to be handled, in the deparsing
> code. And I think it's actually more overhead to qualify every object
> compared to just logging the search_path and enforcing it on the apply
> worker.
>
But doing it in the deparsing code will have the benefit that the
other plugins won't have to develop similar logic.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-08 07:09 Dilip Kumar <dilipbalaut@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Dilip Kumar @ 2022-05-08 07:09 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Sat, May 7, 2022 at 9:38 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, May 6, 2022 at 10:21 PM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > > Attached is a set of two patches as an attempt to evaluate this approach.
> > >
> >
> > Thanks for exploring this direction.
> >
> > I read the deparsing thread and your patch. Here is my thought:
> > 1. The main concern on maintainability of the deparsing code still
> > applies if we want to adapt it for DDL replication.
> >
>
> I agree that it adds to our maintainability effort, like every time we
> enhance any DDL or add a new DDL that needs to be replicated, we
> probably need to change the deparsing code. But OTOH this approach
> seems to provide better flexibility. So, in the long run, maybe the
> effort is worthwhile. I am not completely sure at this stage which
> approach is better but I thought it is worth evaluating this approach
> as Alvaro and Robert seem to prefer this idea.
+1, IMHO with deparsing logic it would be easy to handle the mixed DDL
commands like ALTER TABLE REWRITE. But the only thing is that we will
have to write the deparsing code for all the utility commands so there
will be a huge amount of new code to maintain.
> > 2. The search_path and role still need to be handled, in the deparsing
> > code. And I think it's actually more overhead to qualify every object
> > compared to just logging the search_path and enforcing it on the apply
> > worker.
> >
>
> But doing it in the deparsing code will have the benefit that the
> other plugins won't have to develop similar logic.
Right, this makes sense.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-09 07:16 Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
parent: Dilip Kumar <dilipbalaut@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Bharath Rupireddy @ 2022-05-09 07:16 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Sun, May 8, 2022 at 12:39 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Sat, May 7, 2022 at 9:38 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Fri, May 6, 2022 at 10:21 PM Zheng Li <zhengli10@gmail.com> wrote:
> > >
> > > > Attached is a set of two patches as an attempt to evaluate this approach.
> > > >
> > >
> > > Thanks for exploring this direction.
> > >
> > > I read the deparsing thread and your patch. Here is my thought:
> > > 1. The main concern on maintainability of the deparsing code still
> > > applies if we want to adapt it for DDL replication.
> > >
> >
> > I agree that it adds to our maintainability effort, like every time we
> > enhance any DDL or add a new DDL that needs to be replicated, we
> > probably need to change the deparsing code. But OTOH this approach
> > seems to provide better flexibility. So, in the long run, maybe the
> > effort is worthwhile. I am not completely sure at this stage which
> > approach is better but I thought it is worth evaluating this approach
> > as Alvaro and Robert seem to prefer this idea.
>
> +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> commands like ALTER TABLE REWRITE. But the only thing is that we will
> have to write the deparsing code for all the utility commands so there
> will be a huge amount of new code to maintain.
I haven't gone through the entire thread, just trying to understand
the need of deparsing logic - do we need the DDL query text to be sent
to subscribers? If yes, why can't we WAL log the DDL query text as-is
and the logical decoding can send those to subscribers, as-is or in a
modified form? Or log the DDLs to a separate log on disk?
Regards,
Bharath Rupireddy.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-09 08:56 Amit Kapila <amit.kapila16@gmail.com>
parent: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-05-09 08:56 UTC (permalink / raw)
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Mon, May 9, 2022 at 12:46 PM Bharath Rupireddy
<bharath.rupireddyforpostgres@gmail.com> wrote:
>
> On Sun, May 8, 2022 at 12:39 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > On Sat, May 7, 2022 at 9:38 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > On Fri, May 6, 2022 at 10:21 PM Zheng Li <zhengli10@gmail.com> wrote:
> > > >
> > > > > Attached is a set of two patches as an attempt to evaluate this approach.
> > > > >
> > > >
> > > > Thanks for exploring this direction.
> > > >
> > > > I read the deparsing thread and your patch. Here is my thought:
> > > > 1. The main concern on maintainability of the deparsing code still
> > > > applies if we want to adapt it for DDL replication.
> > > >
> > >
> > > I agree that it adds to our maintainability effort, like every time we
> > > enhance any DDL or add a new DDL that needs to be replicated, we
> > > probably need to change the deparsing code. But OTOH this approach
> > > seems to provide better flexibility. So, in the long run, maybe the
> > > effort is worthwhile. I am not completely sure at this stage which
> > > approach is better but I thought it is worth evaluating this approach
> > > as Alvaro and Robert seem to prefer this idea.
> >
> > +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> > commands like ALTER TABLE REWRITE. But the only thing is that we will
> > have to write the deparsing code for all the utility commands so there
> > will be a huge amount of new code to maintain.
>
> I haven't gone through the entire thread, just trying to understand
> the need of deparsing logic
>
That provided more flexibility, for example, see [1]. We can do things
like identifying the right schema for an object in a way that all
plugins don't need to do specific handling. I think the point Dilip is
highlighting about table rewrite is explained in the email [2]. The
point is that sometimes we can't replay the DDL statement as it is on
the subscriber and deparsing might help in some of those cases even if
not all. At this stage, we are just evaluating both approaches.
[1] - https://www.postgresql.org/message-id/202204081134.6tcmf5cxl3sz%40alvherre.pgsql
[2] - https://www.postgresql.org/message-id/3c646317-df34-4cb3-9365-14abeada6587%40www.fastmail.com
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-09 09:05 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: Dilip Kumar <dilipbalaut@gmail.com>
1 sibling, 2 replies; 449+ messages in thread
From: Alvaro Herrera @ 2022-05-09 09:05 UTC (permalink / raw)
To: Dilip Kumar <dilipbalaut@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On 2022-May-08, Dilip Kumar wrote:
> On Sat, May 7, 2022 at 9:38 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > I agree that it adds to our maintainability effort, like every time we
> > enhance any DDL or add a new DDL that needs to be replicated, we
> > probably need to change the deparsing code. But OTOH this approach
> > seems to provide better flexibility. So, in the long run, maybe the
> > effort is worthwhile. I am not completely sure at this stage which
> > approach is better but I thought it is worth evaluating this approach
> > as Alvaro and Robert seem to prefer this idea.
>
> +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> commands like ALTER TABLE REWRITE. But the only thing is that we will
> have to write the deparsing code for all the utility commands so there
> will be a huge amount of new code to maintain.
Actually, the largest stumbling block on this, IMO, is having a good
mechanism to verify that all DDLs are supported. The deparsing code
itself is typically not very bad, as long as we have a sure way to twist
every contributor's hand into writing it, which is what an automated
test mechanism would give us.
The code in test_ddl_deparse is a pretty lame start, not nearly good
enough by a thousand miles. My real intention was to have a test
harness that would first run a special SQL script to install DDL
capture, then run all the regular src/test/regress scripts, and then at
the end ensure that all the DDL scripts were properly reproduced -- for
example transmit them to another database, replay them there, and dump
both databases and compare them. However, there were challenges which I
no longer remember and we were unable to complete this, and we are where
we are.
Thanks for rebasing that old code, BTW.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 07:02 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-10 07:02 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Euler Taveira <euler@eulerto.com>; japin <japinli@hotmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; rajesh.rs0541@gmail.com, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Apr 13, 2022 at 6:50 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Apr 13, 2022 at 2:38 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > >
> > > > The *initial* DDL replication is a different problem than DDL replication. The
> > > > former requires a snapshot to read the current catalog data and build a CREATE
> > > > command as part of the subscription process. The subsequent DDLs in that object
> > > > will be handled by a different approach that is being discussed here.
> > > >
> > >
> > > I think they are not completely independent because of the current way
> > > to do initial sync followed by replication. The initial sync and
> > > replication need some mechanism to ensure that one of those doesn't
> > > overwrite the work done by the other. Now, the initial idea and patch
> > > can be developed separately but I think both the patches have some
> > > dependency.
> >
> > I agree with the point that their design can not be completely
> > independent. They have some logical relationship of what schema will
> > be copied by the initial sync and where is the exact boundary from
> > which we will start sending as replication. And suppose first we only
> > plan to implement the replication part then how the user will know
> > what all schema user has to create and what will be replicated using
> > DDL replication? Suppose the user takes a dump and copies all the
> > schema and then creates the subscription, then how we are we going to
> > handle the DDL concurrent to the subscription command?
> >
>
> Right, I also don't see how it can be done in the current
> implementation. So, I think even if we want to develop these two as
> separate patches they need to be integrated to make the solution
> complete.
It would be better to develop them separately in terms of development
speed but, yes, we perhaps need to integrate them at some points.
I think that the initial DDL replication can be done when the
relation's state is SUBREL_STATE_INIT. That is, at the very beginning
of the table synchronization, the syncworker copies the table schema
somehow, then starts the initial data copy. After that, syncworker or
applyworker applies DML/DDL changes while catching up and streaming
changes, respectively. Probably we can have it optional whether to
copy schema only, data only, or both.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 09:27 Amit Kapila <amit.kapila16@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-05-10 09:27 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Euler Taveira <euler@eulerto.com>; japin <japinli@hotmail.com>; Zheng Li <zhengli10@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Tue, May 10, 2022 at 12:32 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, Apr 13, 2022 at 6:50 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Apr 13, 2022 at 2:38 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >
> > > On Tue, Apr 12, 2022 at 4:25 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > >
> > > > > The *initial* DDL replication is a different problem than DDL replication. The
> > > > > former requires a snapshot to read the current catalog data and build a CREATE
> > > > > command as part of the subscription process. The subsequent DDLs in that object
> > > > > will be handled by a different approach that is being discussed here.
> > > > >
> > > >
> > > > I think they are not completely independent because of the current way
> > > > to do initial sync followed by replication. The initial sync and
> > > > replication need some mechanism to ensure that one of those doesn't
> > > > overwrite the work done by the other. Now, the initial idea and patch
> > > > can be developed separately but I think both the patches have some
> > > > dependency.
> > >
> > > I agree with the point that their design can not be completely
> > > independent. They have some logical relationship of what schema will
> > > be copied by the initial sync and where is the exact boundary from
> > > which we will start sending as replication. And suppose first we only
> > > plan to implement the replication part then how the user will know
> > > what all schema user has to create and what will be replicated using
> > > DDL replication? Suppose the user takes a dump and copies all the
> > > schema and then creates the subscription, then how we are we going to
> > > handle the DDL concurrent to the subscription command?
> > >
> >
> > Right, I also don't see how it can be done in the current
> > implementation. So, I think even if we want to develop these two as
> > separate patches they need to be integrated to make the solution
> > complete.
>
> It would be better to develop them separately in terms of development
> speed but, yes, we perhaps need to integrate them at some points.
>
> I think that the initial DDL replication can be done when the
> relation's state is SUBREL_STATE_INIT. That is, at the very beginning
> of the table synchronization, the syncworker copies the table schema
> somehow, then starts the initial data copy. After that, syncworker or
> applyworker applies DML/DDL changes while catching up and streaming
> changes, respectively. Probably we can have it optional whether to
> copy schema only, data only, or both.
>
This sounds okay for copying table schema but we can have other
objects like functions, procedures, views, etc. So, we may need
altogether a separate mechanism to copy all the published objects.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 11:33 Ajin Cherian <itsajin@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 3 replies; 449+ messages in thread
From: Ajin Cherian @ 2022-05-10 11:33 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com, Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Fri, May 6, 2022 at 11:24 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> As we have hacked CreatePublication function for this POC, the
> regression tests are not passing but we can easily change it so that
> we invoke new functionality with the syntax proposed in this thread or
> with some other syntax and we shall do that in the next patch unless
> this approach is not worth pursuing.
>
> This POC is prepared by Ajin Cherian, Hou-San, and me.
>
> Thoughts?
>
> [1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
I have updated Amit's patch by including a public action "create" when
creating publication which is turned off by default.
Now the 'make check' tests pass. I also fixed a problem that failed to
create tables when the table has a primary key.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v2-0001-Fix-race-in-032_relfilenode_reuse.pl.patch (1.7K, ../../CAFPTHDbEG-TCAAcUKQTP+hM2rFOCVn4RbkqHm4=9ua-ZY_rRNw@mail.gmail.com/2-v2-0001-Fix-race-in-032_relfilenode_reuse.pl.patch)
download | inline diff:
From a22652ebbcb33a6a271cc69841ad61d971d7db30 Mon Sep 17 00:00:00 2001
From: Thomas Munro <tmunro@postgresql.org>
Date: Sun, 8 May 2022 16:54:09 +1200
Subject: [PATCH v2 1/2] Fix race in 032_relfilenode_reuse.pl.
Add wait_for_catchup() call to the test added by commit e2f65f42. Per
slow build farm animal grison.
Also fix a comment.
Discussion: https://postgr.es/m/CA%2BhUKGLJ2Vy8hVQmnYotmTaEKZK0%3D-GcXgNAgcHzArZvtS4L_g%40mail.gmail.com
---
src/test/recovery/t/032_relfilenode_reuse.pl | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/test/recovery/t/032_relfilenode_reuse.pl b/src/test/recovery/t/032_relfilenode_reuse.pl
index 22d8e85..ac9340b 100644
--- a/src/test/recovery/t/032_relfilenode_reuse.pl
+++ b/src/test/recovery/t/032_relfilenode_reuse.pl
@@ -72,6 +72,8 @@ $node_primary->safe_psql('postgres', q[
CREATE TABLE replace_sb(data text);
INSERT INTO replace_sb(data) SELECT random()::text FROM generate_series(1, 15000);]);
+$node_primary->wait_for_catchup($node_standby);
+
# Use longrunning transactions, so that AtEOXact_SMgr doesn't close files
send_query_and_wait(
\%psql_primary,
@@ -101,9 +103,9 @@ verify($node_primary, $node_standby, 1,
# value so we can check everything is OK
$node_primary->safe_psql('conflict_db', "UPDATE large SET datab = 2;");
-# Again cause a lot of IO. That'll again write back dirty data, but uses (XXX
-# adjust after bugfix) the already opened file descriptor.
-# FIXME
+# Again cause a lot of IO. That'll again write back dirty data, but uses newly
+# opened file descriptors, so we don't confuse old files with new files despite
+# recycling relfilenodes.
cause_eviction(\%psql_primary, \%psql_standby);
verify($node_primary, $node_standby, 2,
--
1.8.3.1
[application/octet-stream] v2-0002-Functions-to-deparse-DDL-commands.patch (63.9K, ../../CAFPTHDbEG-TCAAcUKQTP+hM2rFOCVn4RbkqHm4=9ua-ZY_rRNw@mail.gmail.com/3-v2-0002-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 4fd948bf4bad777cffda13f77a188895d1f9a887 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Sun, 8 May 2022 11:02:40 -0400
Subject: [PATCH v2 2/2] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE specific functions. We can extend
it as we need more functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 1487 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 6 +
src/include/tcop/ddl_deparse.h | 9 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2263 insertions(+)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..ac92f10
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,1487 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 5d49f56..ab2a10b 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2139,6 +2139,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns the constraint definition.
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6d378ff..ba908b3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11881,5 +11881,11 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..e979ee5
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,9 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 12:39 Amit Kapila <amit.kapila16@gmail.com>
parent: Ajin Cherian <itsajin@gmail.com>
2 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-05-10 12:39 UTC (permalink / raw)
To: Ajin Cherian <itsajin@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Tue, May 10, 2022 at 4:59 PM Ajin Cherian <itsajin@gmail.com> wrote:
>
> On Fri, May 6, 2022 at 11:24 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > As we have hacked CreatePublication function for this POC, the
> > regression tests are not passing but we can easily change it so that
> > we invoke new functionality with the syntax proposed in this thread or
> > with some other syntax and we shall do that in the next patch unless
> > this approach is not worth pursuing.
> >
> > This POC is prepared by Ajin Cherian, Hou-San, and me.
> >
> > Thoughts?
> >
> > [1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
>
> I have updated Amit's patch by including a public action "create" when
> creating publication which is turned off by default.
> Now the 'make check' tests pass. I also fixed a problem that failed to
> create tables when the table has a primary key.
>
Are these the correct set of patches? I don't see a DDL support patch.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 15:17 Ajin Cherian <itsajin@gmail.com>
parent: Ajin Cherian <itsajin@gmail.com>
2 siblings, 1 reply; 449+ messages in thread
From: Ajin Cherian @ 2022-05-10 15:17 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com, Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Tue, May 10, 2022 at 9:33 PM Ajin Cherian <itsajin@gmail.com> wrote:
>
> On Fri, May 6, 2022 at 11:24 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > As we have hacked CreatePublication function for this POC, the
> > regression tests are not passing but we can easily change it so that
> > we invoke new functionality with the syntax proposed in this thread or
> > with some other syntax and we shall do that in the next patch unless
> > this approach is not worth pursuing.
> >
> > This POC is prepared by Ajin Cherian, Hou-San, and me.
> >
> > Thoughts?
> >
> > [1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
>
> I have updated Amit's patch by including a public action "create" when
> creating publication which is turned off by default.
> Now the 'make check' tests pass. I also fixed a problem that failed to
> create tables when the table has a primary key.
Sorry, I sent the wrong patches. Here is the correct one.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v2-0001-Functions-to-deparse-DDL-commands.patch (62.6K, ../../CAFPTHDYza7PZqKdkyoRUpJTDC26x_XZtJnFwtiyy9C_HuUS7OA@mail.gmail.com/2-v2-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 0dbd163515453dfabac5ab24f8ace65d42f17aff Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Sun, 8 May 2022 11:02:40 -0400
Subject: [PATCH v2 1/2] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE specific functions. We can extend
it as we need more functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 1487 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++++++++++
src/include/catalog/pg_proc.dat | 6 +
src/include/tcop/ddl_deparse.h | 9 +
5 files changed, 2253 insertions(+)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..bbd21f3
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,1487 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6d378ff..ba908b3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11881,5 +11881,11 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..e979ee5
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,9 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+
+#endif /* DDL_DEPARSE_H */
--
1.8.3.1
[application/octet-stream] v2-0002-Support-DDL-replication.patch (104.5K, ../../CAFPTHDYza7PZqKdkyoRUpJTDC26x_XZtJnFwtiyy9C_HuUS7OA@mail.gmail.com/3-v2-0002-Support-DDL-replication.patch)
download | inline diff:
From 989873ed938637e919dae87df037aebfdb52d912 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Sun, 8 May 2022 11:07:46 -0400
Subject: [PATCH v2 2/2] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to a simple CREATE TABLE command.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 51 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/commands/ddl_deparse.c | 2 +-
src/backend/commands/event_trigger.c | 44 +++
src/backend/commands/publicationcmds.c | 50 +++
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 83 +++++
src/backend/replication/logical/decode.c | 40 +++
src/backend/replication/logical/logical.c | 88 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 123 +++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 39 ++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/adt/ruleutils.c | 9 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/logicalddlmsgdesc.c | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 3 +
src/include/catalog/pg_publication.h | 4 +
src/include/replication/ddlmessage.h | 41 +++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 23 ++
src/include/replication/reorderbuffer.h | 30 ++
src/include/utils/ruleutils.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
33 files changed, 1161 insertions(+), 210 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 120000 src/bin/pg_waldump/logicalddlmsgdesc.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..4dc7591
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,51 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index e1d6ebb..b97bbe9 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index bbd21f3..ac92f10 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -1107,7 +1107,7 @@ obtainConstraints(List *elements, Oid relationId, Oid domainId)
"contype", ObjTypeString, contype,
"name", ObjTypeString, NameStr(constrForm->conname),
"definition", ObjTypeString,
- pg_get_constraintdef_command(constrForm->oid));
+ pg_get_constraintdef_command_simple(constrForm->oid));
elements = lappend(elements, new_object_object(tmp));
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..8446a00 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -2180,3 +2183,44 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+ static int i;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Fixme: This is just for the development purpose and needs to be
+ * removed later.
+ */
+ elog(LOG, "DDL command no. %d", ++i);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", json_string, strlen(json_string) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 6df0e66..f41c22a 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubcreate = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "create") == 0)
+ pubactions->pubcreate = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -755,6 +760,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -819,6 +825,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubcreate - 1] =
+ BoolGetDatum(pubactions.pubcreate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -881,6 +889,45 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (stmt->for_all_tables && pubactions.pubcreate)
+ {
+ CreateEventTrigStmt *ddl_trigg;
+ Node *args = NULL;
+ List *tags = NIL;
+ Oid event_trig_id;
+ char trigger_name[NAMEDATALEN];
+
+ ddl_trigg = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), "pg_deparse_trig_%u",
+ puboid);
+ ddl_trigg->trigname = pstrdup(trigger_name);
+ ddl_trigg->eventname = "ddl_command_end";
+ ddl_trigg->funcname = SystemFuncName("publication_ddl_deparse");
+
+ args = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ tags = list_make1(args);
+ ddl_trigg->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+ event_trig_id = CreateEventTrigger(ddl_trigg);
+
+ /*
+ * Register the event trigger as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, event_trig_id);
+ recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1021,6 +1068,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubcreate - 1] = BoolGetDatum(pubactions.pubcreate);
+ replaces[Anum_pg_publication_pubcreate - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..8e0be0f
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,83 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6303647..14dc9c4 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,45 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 788769d..d01c143 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,9 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +93,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +225,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +242,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +260,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1206,6 +1216,43 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1511,6 +1558,47 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6887dc2..1c04296 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +875,33 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1993,25 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2389,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -3711,6 +3769,36 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4025,6 +4113,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4283,8 +4380,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4300,6 +4396,29 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 7da7823..fc5e42cf 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b197bfd..5ec9c3e 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,9 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -255,6 +258,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -271,6 +275,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1649,8 +1654,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1689,6 +1694,36 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 01d4c22..9649c5b 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 5d49f56..5779fac 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2148,6 +2148,15 @@ pg_get_constraintdef_command(Oid constraintId)
}
/*
+ * Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* As of 9.4, we now use an MVCC snapshot for this.
*/
static char *
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 43f14c2..b850e7c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5634,6 +5634,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubcreate |= pubform->pubcreate;
/*
* Check if all columns referenced in the filter expression are part of
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 786d592..b4ee0f9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubcreate;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubcreate, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubcreate = PQfnumber(res, "pubcreate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubcreate =
+ (strcmp(PQgetvalue(res, i, i_pubcreate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubcreate)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "create");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..170271e 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubcreate;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/logicalddlmsgdesc.c b/src/bin/pg_waldump/logicalddlmsgdesc.c
new file mode 120000
index 0000000..9b627d7
--- /dev/null
+++ b/src/bin/pg_waldump/logicalddlmsgdesc.c
@@ -0,0 +1 @@
+../../../src/backend/access/rmgrdesc/logicalddlmsgdesc.c
\ No newline at end of file
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4369f22..64c2b86 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubcreate AS \"%s\"",
+ gettext_noop("Creates"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubcreate;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubcreate = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubcreate)
+ appendPQExpBufferStr(&buf,
+ ", pubcreate");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubcreate)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubcreate)
+ printTableAddHeader(&cont, gettext_noop("Creates"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubcreate)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ba908b3..7f0d2ca 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11887,5 +11887,8 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 29b1856..f954907 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubcreate;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubcreate;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..bc95f3c
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index a33c2a7..697867a 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..fa7a8b8 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,16 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +210,17 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +240,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +259,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index f12e75d..a6a9133 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,14 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +439,14 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +513,15 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +567,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +587,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +663,8 @@ void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size, const char *message);
void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 1c5b5d2..ed546c6 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+------+-------+------------+---------+---------+---------+-----------+----------+---------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 398c0f3..ff56ece 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-10 15:48 Zheng Li <zhengli10@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-05-10 15:48 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
> > > I agree that it adds to our maintainability effort, like every time we
> > > enhance any DDL or add a new DDL that needs to be replicated, we
> > > probably need to change the deparsing code. But OTOH this approach
> > > seems to provide better flexibility. So, in the long run, maybe the
> > > effort is worthwhile. I am not completely sure at this stage which
> > > approach is better but I thought it is worth evaluating this approach
> > > as Alvaro and Robert seem to prefer this idea.
> >
> > +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> > commands like ALTER TABLE REWRITE. But the only thing is that we will
> > have to write the deparsing code for all the utility commands so there
> > will be a huge amount of new code to maintain.
>
> Actually, the largest stumbling block on this, IMO, is having a good
> mechanism to verify that all DDLs are supported. The deparsing code
> itself is typically not very bad, as long as we have a sure way to twist
> every contributor's hand into writing it, which is what an automated
> test mechanism would give us.
>
> The code in test_ddl_deparse is a pretty lame start, not nearly good
> enough by a thousand miles. My real intention was to have a test
> harness that would first run a special SQL script to install DDL
> capture, then run all the regular src/test/regress scripts, and then at
> the end ensure that all the DDL scripts were properly reproduced -- for
> example transmit them to another database, replay them there, and dump
> both databases and compare them. However, there were challenges which I
> no longer remember and we were unable to complete this, and we are where
> we are.
>
> Thanks for rebasing that old code, BTW.
I agree that deparsing could be a very useful utility on its own. Not
only for SQL command replication between PostgreSQL servers, but also
potentially feasible for SQL command replication between PotgreSQL and
other database systems. For example, one could assemble the json
representation of the SQL parse tree back to a SQL command that can be
run in MySQL. But that requires different assembling rules and code
for different database systems. If we're envisioning this kind of
flexibility that the deparsing utility can offer, then I think it's
better to develop the deparsing utility as an extension itself.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-11 04:45 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-11 04:45 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Wed, May 11, 2022 at 1:01 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> > > > I agree that it adds to our maintainability effort, like every time we
> > > > enhance any DDL or add a new DDL that needs to be replicated, we
> > > > probably need to change the deparsing code. But OTOH this approach
> > > > seems to provide better flexibility. So, in the long run, maybe the
> > > > effort is worthwhile. I am not completely sure at this stage which
> > > > approach is better but I thought it is worth evaluating this approach
> > > > as Alvaro and Robert seem to prefer this idea.
> > >
> > > +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> > > commands like ALTER TABLE REWRITE. But the only thing is that we will
> > > have to write the deparsing code for all the utility commands so there
> > > will be a huge amount of new code to maintain.
> >
> > Actually, the largest stumbling block on this, IMO, is having a good
> > mechanism to verify that all DDLs are supported. The deparsing code
> > itself is typically not very bad, as long as we have a sure way to twist
> > every contributor's hand into writing it, which is what an automated
> > test mechanism would give us.
> >
> > The code in test_ddl_deparse is a pretty lame start, not nearly good
> > enough by a thousand miles. My real intention was to have a test
> > harness that would first run a special SQL script to install DDL
> > capture, then run all the regular src/test/regress scripts, and then at
> > the end ensure that all the DDL scripts were properly reproduced -- for
> > example transmit them to another database, replay them there, and dump
> > both databases and compare them. However, there were challenges which I
> > no longer remember and we were unable to complete this, and we are where
> > we are.
> >
> > Thanks for rebasing that old code, BTW.
>
> I agree that deparsing could be a very useful utility on its own. Not
> only for SQL command replication between PostgreSQL servers, but also
> potentially feasible for SQL command replication between PotgreSQL and
> other database systems. For example, one could assemble the json
> representation of the SQL parse tree back to a SQL command that can be
> run in MySQL. But that requires different assembling rules and code
> for different database systems. If we're envisioning this kind of
> flexibility that the deparsing utility can offer, then I think it's
> better to develop the deparsing utility as an extension itself.
IIUC the event trigger can already provide such flexibility. That is,
one could create an extension that creates an event trigger and in the
event trigger function it deparses the SQL parse tree to a SQL command
that can be run in MySQL. While having such flexibility, I’m fine with
having built-in deparsing utility in the core.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-11 06:21 Ajin Cherian <itsajin@gmail.com>
parent: Ajin Cherian <itsajin@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Ajin Cherian @ 2022-05-11 06:21 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com, Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Wed, May 11, 2022 at 1:17 AM Ajin Cherian <itsajin@gmail.com> wrote:
> > I have updated Amit's patch by including a public action "create" when
> > creating publication which is turned off by default.
> > Now the 'make check' tests pass. I also fixed a problem that failed to
> > create tables when the table has a primary key.
>
I have rearranged the files around and removed
src/bin/pg_waldump/logicalddlmsgdesc.c
as it is a generated file.
regards,
Ajin Cherian
Fujitsu Australia
Attachments:
[application/octet-stream] v3-0001-Functions-to-deparse-DDL-commands.patch (63.9K, ../../CAFPTHDa1kiF8wqCt_z6_v0U-1FHujTjJL0k0YTCRNFSe82wpnw@mail.gmail.com/2-v3-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 9eba6beab1b9b3923fc21cacc8b08f212a2a9e3a Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 11 May 2022 00:43:43 -0400
Subject: [PATCH v2 1/2] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE specific functions. We can extend
it as we need more functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 1487 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 6 +
src/include/tcop/ddl_deparse.h | 9 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2263 insertions(+)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..ac92f10
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,1487 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 5d49f56..4f38f74 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2139,6 +2139,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6d378ff..ba908b3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11881,5 +11881,11 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..e979ee5
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,9 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
1.8.3.1
[application/octet-stream] v3-0002-Support-DDL-replication.patch (102.2K, ../../CAFPTHDa1kiF8wqCt_z6_v0U-1FHujTjJL0k0YTCRNFSe82wpnw@mail.gmail.com/3-v3-0002-Support-DDL-replication.patch)
download | inline diff:
From fe55e489e9cfa30149d9886072d3fd420138093c Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 11 May 2022 02:09:50 -0400
Subject: [PATCH v2 2/2] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to a simple CREATE TABLE command.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 51 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/commands/event_trigger.c | 44 +++
src/backend/commands/publicationcmds.c | 50 +++
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 83 +++++
src/backend/replication/logical/decode.c | 40 +++
src/backend/replication/logical/logical.c | 88 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 123 +++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 39 ++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 3 +
src/include/catalog/pg_publication.h | 4 +
src/include/replication/ddlmessage.h | 41 +++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 23 ++
src/include/replication/reorderbuffer.h | 30 ++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
29 files changed, 1149 insertions(+), 209 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..4dc7591
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,51 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index e1d6ebb..b97bbe9 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..8446a00 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -2180,3 +2183,44 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+ static int i;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Fixme: This is just for the development purpose and needs to be
+ * removed later.
+ */
+ elog(LOG, "DDL command no. %d", ++i);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", json_string, strlen(json_string) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 6df0e66..f41c22a 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubcreate = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "create") == 0)
+ pubactions->pubcreate = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -755,6 +760,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -819,6 +825,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubcreate - 1] =
+ BoolGetDatum(pubactions.pubcreate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -881,6 +889,45 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (stmt->for_all_tables && pubactions.pubcreate)
+ {
+ CreateEventTrigStmt *ddl_trigg;
+ Node *args = NULL;
+ List *tags = NIL;
+ Oid event_trig_id;
+ char trigger_name[NAMEDATALEN];
+
+ ddl_trigg = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), "pg_deparse_trig_%u",
+ puboid);
+ ddl_trigg->trigname = pstrdup(trigger_name);
+ ddl_trigg->eventname = "ddl_command_end";
+ ddl_trigg->funcname = SystemFuncName("publication_ddl_deparse");
+
+ args = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ tags = list_make1(args);
+ ddl_trigg->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+ event_trig_id = CreateEventTrigger(ddl_trigg);
+
+ /*
+ * Register the event trigger as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, event_trig_id);
+ recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1021,6 +1068,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubcreate - 1] = BoolGetDatum(pubactions.pubcreate);
+ replaces[Anum_pg_publication_pubcreate - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..8e0be0f
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,83 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 6303647..14dc9c4 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,45 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 788769d..d01c143 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,9 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +93,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +225,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +242,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +260,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1206,6 +1216,43 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1511,6 +1558,47 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6887dc2..1c04296 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +875,33 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1993,25 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2389,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -3711,6 +3769,36 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4025,6 +4113,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4283,8 +4380,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4300,6 +4396,29 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 7da7823..fc5e42cf 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index b197bfd..5ec9c3e 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,9 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -255,6 +258,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -271,6 +275,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1649,8 +1654,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1689,6 +1694,36 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 01d4c22..9649c5b 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 43f14c2..b850e7c 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5634,6 +5634,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubcreate |= pubform->pubcreate;
/*
* Check if all columns referenced in the filter expression are part of
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 786d592..b4ee0f9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubcreate;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubcreate, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubcreate = PQfnumber(res, "pubcreate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubcreate =
+ (strcmp(PQgetvalue(res, i, i_pubcreate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubcreate)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "create");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..170271e 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubcreate;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 31df8b7..2814a13 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubcreate AS \"%s\"",
+ gettext_noop("Creates"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubcreate;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubcreate = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubcreate)
+ appendPQExpBufferStr(&buf,
+ ", pubcreate");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubcreate)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubcreate)
+ printTableAddHeader(&cont, gettext_noop("Creates"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubcreate)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ba908b3..7f0d2ca 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11887,5 +11887,8 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 29b1856..f954907 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubcreate;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubcreate;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..bc95f3c
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,41 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index a33c2a7..697867a 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..fa7a8b8 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,16 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +210,17 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +240,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +259,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index f12e75d..a6a9133 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,14 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +439,14 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +513,15 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +567,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +587,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +663,8 @@ void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size, const char *message);
void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 1c5b5d2..ed546c6 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+------+-------+------------+---------+---------+---------+-----------+----------+---------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 398c0f3..ff56ece 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-11 07:38 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
1 sibling, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-11 07:38 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Mon, May 9, 2022 at 6:05 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2022-May-08, Dilip Kumar wrote:
>
> > On Sat, May 7, 2022 at 9:38 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > > I agree that it adds to our maintainability effort, like every time we
> > > enhance any DDL or add a new DDL that needs to be replicated, we
> > > probably need to change the deparsing code. But OTOH this approach
> > > seems to provide better flexibility. So, in the long run, maybe the
> > > effort is worthwhile. I am not completely sure at this stage which
> > > approach is better but I thought it is worth evaluating this approach
> > > as Alvaro and Robert seem to prefer this idea.
> >
> > +1, IMHO with deparsing logic it would be easy to handle the mixed DDL
> > commands like ALTER TABLE REWRITE. But the only thing is that we will
> > have to write the deparsing code for all the utility commands so there
> > will be a huge amount of new code to maintain.
>
> Actually, the largest stumbling block on this, IMO, is having a good
> mechanism to verify that all DDLs are supported. The deparsing code
> itself is typically not very bad, as long as we have a sure way to twist
> every contributor's hand into writing it, which is what an automated
> test mechanism would give us.
Agreed.
>
> The code in test_ddl_deparse is a pretty lame start, not nearly good
> enough by a thousand miles. My real intention was to have a test
> harness that would first run a special SQL script to install DDL
> capture, then run all the regular src/test/regress scripts, and then at
> the end ensure that all the DDL scripts were properly reproduced -- for
> example transmit them to another database, replay them there, and dump
> both databases and compare them. However, there were challenges which I
> no longer remember and we were unable to complete this, and we are where
> we are.
I think the regression test suite improvements in these few years make
it easier to implement such regression tests in order to check not
only the existing commands but also future changes. I'll try it if no
one is working on it, and let us see if there are challenges.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-11 09:15 Amit Kapila <amit.kapila16@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-05-11 09:15 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Wed, May 11, 2022 at 1:09 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Mon, May 9, 2022 at 6:05 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> >
> > On 2022-May-08, Dilip Kumar wrote:
>
> >
> > The code in test_ddl_deparse is a pretty lame start, not nearly good
> > enough by a thousand miles. My real intention was to have a test
> > harness that would first run a special SQL script to install DDL
> > capture, then run all the regular src/test/regress scripts, and then at
> > the end ensure that all the DDL scripts were properly reproduced -- for
> > example transmit them to another database, replay them there, and dump
> > both databases and compare them. However, there were challenges which I
> > no longer remember and we were unable to complete this, and we are where
> > we are.
>
> I think the regression test suite improvements in these few years make
> it easier to implement such regression tests in order to check not
> only the existing commands but also future changes. I'll try it if no
> one is working on it, and let us see if there are challenges.
>
I think it is a good idea to give it a try. However, one point to note
in this regard is that if we decide to use deparsing for DDL logical
replication then the tests for logical replication will automatically
test all the deparsing code.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-11 12:55 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-11 12:55 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Wed, May 11, 2022 at 6:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, May 11, 2022 at 1:09 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > On Mon, May 9, 2022 at 6:05 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > >
> > > On 2022-May-08, Dilip Kumar wrote:
> >
> > >
> > > The code in test_ddl_deparse is a pretty lame start, not nearly good
> > > enough by a thousand miles. My real intention was to have a test
> > > harness that would first run a special SQL script to install DDL
> > > capture, then run all the regular src/test/regress scripts, and then at
> > > the end ensure that all the DDL scripts were properly reproduced -- for
> > > example transmit them to another database, replay them there, and dump
> > > both databases and compare them. However, there were challenges which I
> > > no longer remember and we were unable to complete this, and we are where
> > > we are.
> >
> > I think the regression test suite improvements in these few years make
> > it easier to implement such regression tests in order to check not
> > only the existing commands but also future changes. I'll try it if no
> > one is working on it, and let us see if there are challenges.
> >
>
> I think it is a good idea to give it a try. However, one point to note
> in this regard is that if we decide to use deparsing for DDL logical
> replication then the tests for logical replication will automatically
> test all the deparsing code.
Do you mean that in tests for logical replication we run regression
tests on the publisher and check if relations are
created/altered/dropped expectedly on the subscriber? If we rely on
tests for logical replication, I think logical replication will have
to support all future DDL changes/features.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-12 02:54 Amit Kapila <amit.kapila16@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-05-12 02:54 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>; Ajin Cherian <itsajin@gmail.com>; Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Wed, May 11, 2022 at 6:25 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Wed, May 11, 2022 at 6:15 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, May 11, 2022 at 1:09 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > On Mon, May 9, 2022 at 6:05 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
> > > >
> > > >
> > > > The code in test_ddl_deparse is a pretty lame start, not nearly good
> > > > enough by a thousand miles. My real intention was to have a test
> > > > harness that would first run a special SQL script to install DDL
> > > > capture, then run all the regular src/test/regress scripts, and then at
> > > > the end ensure that all the DDL scripts were properly reproduced -- for
> > > > example transmit them to another database, replay them there, and dump
> > > > both databases and compare them. However, there were challenges which I
> > > > no longer remember and we were unable to complete this, and we are where
> > > > we are.
> > >
> > > I think the regression test suite improvements in these few years make
> > > it easier to implement such regression tests in order to check not
> > > only the existing commands but also future changes. I'll try it if no
> > > one is working on it, and let us see if there are challenges.
> > >
> >
> > I think it is a good idea to give it a try. However, one point to note
> > in this regard is that if we decide to use deparsing for DDL logical
> > replication then the tests for logical replication will automatically
> > test all the deparsing code.
>
> Do you mean that in tests for logical replication we run regression
> tests on the publisher and check if relations are
> created/altered/dropped expectedly on the subscriber?
>
Yes.
> If we rely on
> tests for logical replication, I think logical replication will have
> to support all future DDL changes/features.
>
Agreed, so we can't completely rely on logical replication tests.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-13 21:01 Zheng Li <zhengli10@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-05-13 21:01 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> > 4. The handling related to partition tables seems missing because, on
> > the subscriber-side, it always creates a relation entry in
> > pg_subscription_rel which won't work. Check its interaction with
> > publish_via_partition_root.
>
> I will test it out.
Hi,
patch 0010 properly handles partitioned table creation on the apply
worker. Whether a replicated partitioned table should be added to
pg_subscription_rel catalog depends on the setting of
publish_via_partition_root of the publication. Thus we need to connect
to the source DB and check if the partitioned table is in
pg_catalog.pg_publication_tables after the apply worker creates the
partitioned table.
Thanks to Borui Yang for enabling and testing replication of DDL type
T_RenameStmt in patch 0009.
I've also rebased all the patches. Github branch of the same change
can be found here:
https://github.com/zli236/postgres/commits/ddl_replication
Regards,
Zheng
Attachments:
[application/octet-stream] 0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch (8.0K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/2-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch)
download | inline diff:
From 1b14cc43a16a3bd0296f7b1564707b9ac44b9de4 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Thu, 24 Mar 2022 17:46:55 +0000
Subject: [PATCH 05/10] Support replication of CREATE ... AS .. and SELECT ...
INTO ... statements.
The idea is to force skipping the direct data population
(which could potentially cause data mismatch compared to the publisher)
with these command on the subscriber by force setting the skipData flag
in the intoClause of the parsetree after the logical replication worker
parses the command. The data sync will be taken care of by DML replication
after the replication of the DDL command.
---
src/backend/replication/logical/worker.c | 51 ++++++++++++++++++++----
src/backend/tcop/utility.c | 41 +++++++++++--------
src/test/subscription/t/030_rep_ddls.pl | 24 +++++++++++
3 files changed, 92 insertions(+), 24 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index c25e3a4cfe..905f47faf5 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2542,23 +2542,58 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
Portal portal;
DestReceiver *receiver;
bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE stmt only */
- char *relname = NULL; /* For CREATE TABLE stmt only */
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
commandTag = CreateCommandTag((Node *)command);
/*
- * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
- * execute the stmt. At that point, CreateStmt may have been freeed up.
+ * execute the stmt. At that point, command->stmt may have been freeed up.
*/
if (commandTag == CMDTAG_CREATE_TABLE)
{
- CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
}
+ else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ RangeVar *rv = castmt->into->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ /* SELECT INTO */
+ else if (commandTag == CMDTAG_SELECT)
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ RangeVar *rv = sstmt->intoClause->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
@@ -2657,12 +2692,12 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
* Table created by DDL replication (database level) is automatically
* added to the subscription here.
*
- * Call AddSubscriptionRelState for CREATE TABEL command to set
- * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+ * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
* new table can be replicated without having to manually run
* "alter subscription ... refresh publication"
*/
- if (commandTag == CMDTAG_CREATE_TABLE)
+ if (relname != NULL)
{
Oid relid;
Oid relnamespace = InvalidOid;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index fdda1385c3..34c7db8cc9 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1142,24 +1142,31 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/*
* CreateTableAsStmt can create either a table a materialized view
- * and they are handled differently.
*/
case T_CreateTableAsStmt:
{
CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+
switch(stmt->objtype)
{
+ /*
+ * Either CREATE TABLE AS or SELECT ... INTO stmt
+ * The statement is logged as is, but when we apply the
+ * CREATE TABLE AS or SELECT ... INTO statemtns on the logical
+ * replication worker, we will force the skipData flag in the
+ * intoClause. This way we avoid direct data population on the
+ * subsriber with the execution of these commands which can
+ * potentially cause data mismatch bewteen the publisher.
+ *
+ * The data sync will be handled by DML replication after the
+ * target table has been created.
+ */
case OBJECT_TABLE:
- /*
- * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
- * 1. A normal CREATE TABLE string that get's logged and replicated via
- * DDL replication.
- * 2. Insertions that get replicated by DML replication.
- */
- break;
+
+ /* CREATE MATERIALIZED VIEW */
case OBJECT_MATVIEW:
/*
- * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * Log these stmt for logical replication if
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
@@ -1304,8 +1311,17 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
if (isCompleteQuery)
+ {
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+ }
+
switch (nodeTag(parsetree))
{
/*
@@ -2135,13 +2151,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a complete top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 562efe2cf7..3b15c6d9f0 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -244,6 +244,30 @@ $node_publisher->wait_for_catchup('mysub');
$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+# TEST CREATE TABLE AS stmt
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t3 AS SELECT a, b from s1.t1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t3;");
+is($result, qq(2), 'CREATE TABLE s1.t3 AS is replicated with data');
+
+# TEST CREATE TABLE AS stmt ... WITH NO DATA
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t4 AS SELECT a, b from s1.t1 WITH NO DATA;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t4;");
+is($result, qq(0), 'CREATE TABLE s1.t4 AS is replicated with no data');
+
+# TEST SELECT INTO stmt
+$node_publisher->safe_psql('postgres', "SELECT b into s1.t6 from s1.t1 where a > 1");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
+is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0002-Support-logical-logging-and-decoding-of-DDL-command-.patch (61.4K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/3-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch)
download | inline diff:
From 3128c125b0d2beac4874f8f433fe2d9915ff8f6f Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:57:23 +0000
Subject: [PATCH 02/10] Support logical logging and decoding of DDL command
string.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current consumers.
The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on
the configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
Support logical decoding of the new WAL record xl_logical_ddl_message.
This is similar to the logical decoding of xl_logical_message. Tests
for this change are added in the test_decoding plugin.
---
contrib/test_decoding/Makefile | 4 +-
.../test_decoding/expected/ddlmessages.out | 42 ++++
contrib/test_decoding/sql/ddlmessages.sql | 28 +++
contrib/test_decoding/test_decoding.c | 61 +++++-
src/backend/access/rmgrdesc/Makefile | 1 +
.../access/rmgrdesc/logicalddlmsgdesc.c | 54 +++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 52 +++++
src/backend/commands/tablecmds.c | 43 +++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 99 +++++++++
src/backend/replication/logical/decode.c | 56 +++++
src/backend/replication/logical/logical.c | 91 ++++++++
.../replication/logical/reorderbuffer.c | 204 +++++++++++++++++-
src/backend/tcop/utility.c | 201 ++++++++++++++++-
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/include/access/rmgrlist.h | 1 +
src/include/commands/tablecmds.h | 3 +-
src/include/replication/ddlmessage.h | 47 ++++
src/include/replication/decode.h | 1 +
src/include/replication/output_plugin.h | 29 +++
src/include/replication/reorderbuffer.h | 39 ++++
src/test/regress/expected/publication.out | 16 +-
23 files changed, 1054 insertions(+), 21 deletions(-)
create mode 100644 contrib/test_decoding/expected/ddlmessages.out
create mode 100644 contrib/test_decoding/sql/ddlmessages.sql
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index b220906479..e58a76d3f1 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -5,7 +5,9 @@ PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- spill slot truncate stream stats twophase twophase_stream
+ spill slot truncate stream stats twophase twophase_stream \
+ ddlmessages
+
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
new file mode 100644
index 0000000000..79284f9def
--- /dev/null
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -0,0 +1,42 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+(6 rows)
+
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
new file mode 100644
index 0000000000..211497ee22
--- /dev/null
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -0,0 +1,28 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index 3736da6784..a44e1f79e3 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -76,6 +76,11 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
TransactionId xid,
const char *gid);
@@ -116,6 +121,11 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
int nrelations, Relation relations[],
@@ -141,6 +151,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
cb->message_cb = pg_decode_message;
+ cb->ddlmessage_cb = pg_decode_ddlmessage;
cb->filter_prepare_cb = pg_decode_filter_prepare;
cb->begin_prepare_cb = pg_decode_begin_prepare_txn;
cb->prepare_cb = pg_decode_prepare_txn;
@@ -153,6 +164,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pg_decode_stream_commit;
cb->stream_change_cb = pg_decode_stream_change;
cb->stream_message_cb = pg_decode_stream_message;
+ cb->stream_ddlmessage_cb = pg_decode_stream_ddlmessage;
cb->stream_truncate_cb = pg_decode_stream_truncate;
}
@@ -747,7 +759,8 @@ pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
appendStringInfo(ctx->out, "message: transactional: %d prefix: %s, sz: %zu content:",
@@ -756,6 +769,19 @@ pg_decode_message(LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+ appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_stream_start(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
@@ -936,7 +962,8 @@ pg_decode_stream_change(LogicalDecodingContext *ctx,
static void
pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
@@ -948,7 +975,35 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
else
{
appendStringInfo(ctx->out, "streaming message: transactional: %d prefix: %s, sz: %zu content:",
- transactional, prefix, sz);
+ transactional, prefix, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ }
+
+ OutputPluginWrite(ctx, true);
+}
+
+/*
+ * In streaming mode, we don't display the contents for transactional messages
+ * as the transaction can abort at a later point in time. We don't want users to
+ * see the message contents until the transaction is committed.
+ */
+static void
+pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char * role, const char * search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+
+ if (transactional)
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
+ transactional, prefix, role, search_path, sz);
+ }
+ else
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
}
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72fd86..b8e29e8df3 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000000..7a352d540a
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *role = xlrec->message + xlrec->prefix_size;
+ char *search_path = xlrec->message + xlrec->prefix_size + xlrec->role_size;
+ char *message = xlrec->message + xlrec->prefix_size + xlrec->role_size + xlrec->search_path_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
+ xlrec->transactional ? "transactional" : "non-transactional",
+ prefix, role, search_path, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed69244e3..6db9a593b8 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -29,6 +29,7 @@
#include "miscadmin.h"
#include "replication/decode.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "storage/standby.h"
#include "utils/builtins.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 71a259153c..92d2763e83 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1156,3 +1156,55 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
+
+/*
+ * Checks if DDL on relation (relid) need xlog for logical replication
+ */
+bool
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+{
+ List *allTablePubs = NIL;
+ List *tablePubs = NIL;
+ ListCell *lc;
+
+ /* Only replicate toplevel DDL command */
+ if (!isTopLevel)
+ return false;
+ if (relid == InvalidOid && !forAllTabPubOnly)
+ return false;
+
+ /*
+ * Log the DDL command if
+ * there is any FOR ALL TABLES publication with pubddl_database on
+ * or
+ * this TABLE belongs to any non FOR ALL publications with pubddl_table on
+ */
+ allTablePubs = GetAllTablesPublications();
+ foreach(lc, allTablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_database)
+ return true;
+ }
+
+ /*
+ * If forAllTabPubOnly is true (i.e. database level replication is required for the DDL
+ * to be logged), we can bail now since no publication has been found with pubddl_database on
+ */
+ if (forAllTabPubOnly)
+ return false;
+
+ tablePubs = GetRelationPublications(relid);
+ foreach(lc, tablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_table)
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..6d1487951f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -80,6 +80,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -1332,13 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(DropStmt *drop)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
+ bool ddlxlog = XLogLogicalInfoActive();
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1437,10 +1439,37 @@ RemoveRelations(DropStmt *drop)
/* Not there? */
if (!OidIsValid(relOid))
{
+ ddlxlog = false;
DropErrorMsgNonExistent(rel, relkind, drop->missing_ok);
continue;
}
+ /*
+ * Only log DROP RELATION cmd for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on or
+ * every relation to be dropped belongs to any non FOR ALL publications with pubddl_table on
+ */
+ if (ddlxlog)
+ {
+ Oid tableOid = InvalidOid;
+
+ if (relkind == RELKIND_RELATION)
+ tableOid = relOid;
+ else if (relkind == RELKIND_INDEX)
+ tableOid = IndexGetRelation(relOid, true);
+ /*
+ * Other relation types require database level ddl replication and are
+ * already logged in LogLogicalDDLCommand() if needed.
+ */
+ else
+ ddlxlog = false;
+
+ /* DROP RELATION or INDEX are allowed in table level DDL replication */
+ if (tableOid != InvalidOid &&
+ !ddl_need_xlog(tableOid, false, isTopLevel))
+ ddlxlog = false;
+ }
+
/*
* Decide if concurrent mode needs to be used here or not. The
* callback retrieved the rel's persistence for us.
@@ -1484,6 +1513,18 @@ RemoveRelations(DropStmt *drop)
add_exact_object_address(&obj, objects);
}
+ /* Log the Drop command for logical replication */
+ if (ddlxlog)
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
performMultipleDeletions(objects, drop->behavior, flags);
free_object_addresses(objects);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fdeb71..f3eeb67312 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000000..f93573079a
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
+ * Transactional messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ * Non-transactional messages are sent to the plugin at the time when the
+ * logical decoding reads them from XLOG. This also means that transactional
+ * messages won't be delivered if the transaction was rolled back but the
+ * non-transactional one will always be delivered.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
+{
+ xl_logical_ddl_message xlrec;
+ const char *role;
+
+ role = GetUserNameFromId(roleoid, false);
+
+ /*
+ * Force xid to be allocated if we're emitting a transactional message.
+ */
+ if (transactional)
+ {
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+ }
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.transactional = transactional;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.role_size = strlen(role) + 1;
+ xlrec.search_path_size = strlen(namespace_search_path) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, role), xlrec.role_size);
+ XLogRegisterData(namespace_search_path, xlrec.search_path_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427ba73..034c7f2413 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -603,6 +604,61 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
message->message + message->prefix_size);
}
+/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ Snapshot snapshot;
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (message->transactional &&
+ !SnapBuildProcessChange(builder, xid, buf->origptr))
+ return;
+ else if (!message->transactional &&
+ (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
+ SnapBuildXactNeedsSkip(builder, buf->origptr)))
+ return;
+
+ snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ message->transactional,
+ message->message,
+ /* first part of message is prefix */
+ message->message + message->prefix_size,
+ /* Second part of message is role*/
+ message->message + message->prefix_size + message->role_size,
+ /* Third part of message is search_path */
+ message->message_size,
+ message->message + message->prefix_size +
+ message->role_size + message->search_path_size);
+}
+
/*
* Consolidated commit record handling between the different form of commit
* records.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4273..3004f02433 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +94,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +226,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +243,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +261,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1220,6 +1231,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char *search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
@@ -1535,6 +1584,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char* search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index da7bd1321c..a69deb6420 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,20 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.role != NULL)
+ pfree(change->data.ddlmsg.role);
+ change->data.ddlmsg.role = NULL;
+ if (change->data.ddlmsg.search_path != NULL)
+ pfree(change->data.ddlmsg.search_path);
+ change->data.ddlmsg.search_path = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -866,6 +880,64 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
}
+/*
+ * A transactional DDL message is queued to be processed upon commit and a
+ * non-transactional DDL message gets processed immediately.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
+{
+ if (transactional)
+ {
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ else
+ {
+ ReorderBufferTXN *txn = NULL;
+ volatile Snapshot snapshot_now = snapshot;
+
+ if (xid != InvalidTransactionId)
+ txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+
+ /* setup snapshot to allow catalog access */
+ SetupHistoricSnapshot(snapshot_now, NULL);
+ PG_TRY();
+ {
+ rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+
+ TeardownHistoricSnapshot(false);
+ }
+ PG_CATCH();
+ {
+ TeardownHistoricSnapshot(true);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+}
+
/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
@@ -1957,6 +2029,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message);
}
+/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
@@ -2335,6 +2430,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(
@@ -3709,6 +3808,53 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the role including the size */
+ memcpy(data, &role_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.role,
+ role_size);
+ data += role_size;
+
+ /* write the search_path including the size */
+ memcpy(data, &search_path_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.search_path,
+ search_path_size);
+ data += search_path_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4023,6 +4169,18 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4283,8 +4441,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4298,6 +4455,49 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+ Size role_size;
+ Size search_path_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read role */
+ memcpy(&role_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.role = MemoryContextAlloc(rb->context,
+ role_size);
+ memcpy(change->data.ddlmsg.role, data, role_size);
+ Assert(change->data.ddlmsg.role[role_size - 1] == '\0');
+ data += role_size;
+
+ /* read search_path */
+ memcpy(&search_path_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.search_path = MemoryContextAlloc(rb->context,
+ search_path_size);
+ memcpy(change->data.ddlmsg.search_path, data, search_path_size);
+ Assert(change->data.ddlmsg.search_path[search_path_size - 1] == '\0');
+ data += search_path_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 0e7b7b3138..849b6d4f7a 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -62,6 +62,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
#include "storage/fd.h"
@@ -86,7 +87,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -987,7 +988,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel);
}
break;
@@ -1087,6 +1088,154 @@ standard_ProcessUtility(PlannedStmt *pstmt,
CommandCounterIncrement();
}
+/*
+ * Log a DDL command for logical replication
+ * Some DDLs are only replicated in Database Level DDL replication
+ * Some can be replicated in Table Level DDL replication.
+ *
+ * Currently we focus on supporting Database Level DDL replication
+ */
+static void
+LogLogicalDDLCommand(Node *parsetree, const char *queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* Fisrtly, commands that are only supported in Database level DDL replication */
+ case T_CreateSchemaStmt:
+ case T_CreateStmt:
+ case T_CreateForeignTableStmt:
+ case T_AlterDomainStmt:
+ case T_DefineStmt:
+ case T_CompositeTypeStmt:
+ case T_CreateEnumStmt:
+ case T_CreateRangeStmt:
+ case T_AlterEnumStmt:
+ case T_ViewStmt:
+ case T_CreateFunctionStmt:
+ case T_AlterFunctionStmt:
+ case T_CreateTrigStmt:
+ case T_CreateDomainStmt:
+ case T_CreateCastStmt:
+ case T_CreateOpClassStmt:
+ case T_CreateOpFamilyStmt:
+ case T_AlterOpFamilyStmt:
+ case T_AlterOperatorStmt:
+ case T_AlterTypeStmt:
+ case T_GrantStmt:
+ case T_AlterCollationStmt:
+ /*
+ * Log these stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+
+ /*
+ * Secondly, commands that may be allowed in Table level DDL replication.
+ * These are currently handled in the later execution path of the command.
+ * Because we need to get the relation id which readily available in later
+ * code path.
+ */
+ case T_AlterTableStmt:
+ case T_IndexStmt:
+ case T_RenameStmt: /* TODO */
+ case T_AlterOwnerStmt: /* TODO */
+ break;
+
+ /* DropStmt depends on the removeType */
+ case T_DropStmt:
+ {
+ DropStmt* stmt = (DropStmt*) parsetree;
+ switch (stmt->removeType)
+ {
+ /* Maybe allowed in Table level DDL replication, handled in later code path */
+ case OBJECT_INDEX:
+ case OBJECT_TABLE:
+ break;
+ /* Drop of sequence is by logical replication of sequences separately */
+ case OBJECT_SEQUENCE:
+ break;
+ /* Drop of other objects are allowed in Database level DDL replication only */
+ case OBJECT_VIEW:
+ case OBJECT_MATVIEW:
+ case OBJECT_FOREIGN_TABLE:
+ default:
+ /*
+ * Log these DropStmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+ }
+ }
+ /*
+ * Lastly, rule out DDLs we don't replicate yet in DDL replication
+ * Some of these can be supported, we just need to investigate and run tests.
+ */
+ case T_CreateExtensionStmt:
+ case T_AlterExtensionStmt:
+ case T_AlterExtensionContentsStmt:
+ case T_CreateFdwStmt:
+ case T_AlterFdwStmt:
+ case T_CreateForeignServerStmt:
+ case T_AlterForeignServerStmt:
+ case T_CreateUserMappingStmt:
+ case T_AlterUserMappingStmt:
+ case T_DropUserMappingStmt:
+ case T_ImportForeignSchemaStmt:
+ case T_RuleStmt:
+ case T_CreateSeqStmt:
+ case T_AlterSeqStmt:
+ case T_CreateTableAsStmt:
+ case T_RefreshMatViewStmt:
+ case T_CreatePLangStmt:
+ case T_CreateConversionStmt:
+ case T_CreateTransformStmt:
+ case T_AlterTSDictionaryStmt:
+ case T_AlterTSConfigurationStmt:
+ case T_AlterTableMoveAllStmt:
+ case T_AlterObjectDependsStmt:
+ case T_AlterObjectSchemaStmt:
+ case T_CommentStmt:
+ case T_DropOwnedStmt:
+ case T_AlterDefaultPrivilegesStmt:
+ case T_CreatePolicyStmt:
+ case T_AlterPolicyStmt:
+ case T_SecLabelStmt:
+ case T_CreateAmStmt:
+ case T_CreatePublicationStmt:
+ case T_AlterPublicationStmt:
+ case T_CreateSubscriptionStmt:
+ case T_AlterSubscriptionStmt:
+ case T_DropSubscriptionStmt:
+ case T_CreateStatsStmt:
+ case T_AlterStatsStmt:
+ break;
+ default:
+ break;
+ }
+}
+
/*
* The "Slow" variant of ProcessUtility should only receive statements
* supported by the event triggers facility. Therefore, we always
@@ -1119,6 +1268,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
switch (nodeTag(parsetree))
{
/*
@@ -1321,6 +1477,23 @@ ProcessUtilitySlow(ParseState *pstate,
EventTriggerAlterTableStart(parsetree);
EventTriggerAlterTableRelid(relid);
+ /*
+ * Log the ALTER TABLE command if
+ * There is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
/* ... and do it */
AlterTable(atstmt, lockmode, &atcontext);
@@ -1539,6 +1712,24 @@ ProcessUtilitySlow(ParseState *pstate,
/* ... and do it */
EventTriggerAlterTableStart(parsetree);
+
+ /*
+ * Log CREATE INDEX cmd for logical replication if
+ * there is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on.
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
address =
DefineIndex(relid, /* OID of heap relation */
stmt,
@@ -1755,7 +1946,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt((DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -1976,7 +2167,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
{
switch (stmt->removeType)
{
@@ -1991,7 +2182,7 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(stmt);
+ RemoveRelations(pstate, stmt, isTopLevel);
break;
default:
RemoveObjects(stmt);
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17bb4c..792f438959 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721c97..9de3b8f2eb 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 5d4037f26e..68781365de 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -18,6 +18,7 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "storage/lock.h"
#include "utils/relcache.h"
@@ -27,7 +28,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(DropStmt *drop);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000000..1e8ef22296
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ bool transactional; /* is message transactional? */
+ Size prefix_size; /* length of prefix */
+ Size role_size; /* length of the role that executes the DDL command */
+ Size search_path_size; /* length of the search path */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ * and null-terminated role of length role_size
+ * and null-terminated search_path of length search_path_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
+ size_t size, bool transactional);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65cf7..427a7b997d 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e697..5b1c245b72 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -88,6 +88,19 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
Size message_size,
const char *message);
+/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Filter changes by origin.
*/
@@ -199,6 +212,20 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
Size message_size,
const char *message);
+/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Callback for streaming truncates from in-progress transactions.
*/
@@ -219,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f877e5..dd89e08efc 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ char *role;
+ char *search_path;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +441,17 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +518,18 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size message_size, const char *message);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 180ebd359e..81cdee0527 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | t | f | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
--
2.32.0
[application/octet-stream] 0003-Logical-replication-of-DDL-messages.patch (41.2K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/4-0003-Logical-replication-of-DDL-messages.patch)
download | inline diff:
From c04442745bbe684f2205f65814208e9b364d82ae Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 17:17:39 +0000
Subject: [PATCH 03/10] Logical replication of DDL messages
Integration with pgoutput:
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced
for this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
Logical replication worker change:
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to SUBREL_STATE_INIT,
So that DML replication can progress on this new table without
manually running "ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
TAP test:
A new TAP test 030_rep_ddl.pl is added. We mainly focused on
testing the happy path of database level replication so far.
Corner case DDLs and table level DDL replication are still to be
carefully tested.
---
.../test_decoding/expected/ddlmessages.out | 26 +-
contrib/test_decoding/sql/ddlmessages.sql | 5 +-
src/backend/replication/logical/proto.c | 63 ++++-
src/backend/replication/logical/worker.c | 264 ++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 82 +++++-
src/backend/tcop/utility.c | 14 +-
src/include/replication/logicalproto.h | 10 +-
src/test/subscription/t/004_sync.pl | 2 +-
src/test/subscription/t/006_rewrite.pl | 2 +-
src/test/subscription/t/008_diff_schema.pl | 2 +-
src/test/subscription/t/009_matviews.pl | 2 +-
src/test/subscription/t/012_collation.pl | 2 +-
src/test/subscription/t/013_partition.pl | 8 +-
src/test/subscription/t/030_rep_ddls.pl | 237 ++++++++++++++++
14 files changed, 684 insertions(+), 35 deletions(-)
create mode 100644 src/test/subscription/t/030_rep_ddls.pl
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 79284f9def..0376f36c24 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -2,6 +2,9 @@
SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
?column?
----------
@@ -20,23 +23,22 @@ BEGIN;
CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
-SELECT pg_drop_replication_slot('regression_slot');
-DROP TABLE test_ddlmessage;
-DROP publication mypub;
- data
---------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
(1 row)
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index 211497ee22..c23610f9b4 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -3,6 +3,10 @@ SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
+
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
CREATE TABLE test_ddlmessage (id serial unique, data int);
@@ -20,7 +24,6 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e2d2..3cfa94dd8c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -662,6 +662,63 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
pq_sendbytes(out, message, sz);
}
+/*
+ * Read DDL MESSAGE from stream
+ */
+const char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ const char **role,
+ const char **search_path,
+ bool *transactional,
+ Size *sz)
+{
+ uint8 flags;
+ const char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *role = pq_getmsgstring(in);
+ *search_path = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* encode and send message flags */
+ if (transactional)
+ flags |= MESSAGE_TRANSACTIONAL;
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendstring(out, role);
+ pq_sendstring(out, search_path);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
/*
* Write relation description to the output stream.
*/
@@ -1218,6 +1275,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9e7b..c25e3a4cfe 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -180,6 +181,8 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "tcop/pquery.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -346,6 +349,10 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
+static void apply_execute_sql_command(const char *cmdstr,
+ const char* role,
+ const char* search_path,
+ bool isTopLevel);
/* Compute GID for two_phase transactions */
static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
@@ -2445,6 +2452,259 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/*
+ * Handle generic messages.
+ */
+static void
+apply_handle_ddlmessage(StringInfo s)
+{
+ XLogRecPtr lsn;
+ bool transactional;
+ Size sz;
+ const char *prefix;
+ const char *role;
+ const char *search_path;
+ const char *msg;
+
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+
+ apply_execute_sql_command(msg, role, search_path, true);
+}
+
+/*
+ * Add context to the errors produced by apply_execute_sql_command().
+ */
+static void
+execute_sql_command_error_cb(void *arg)
+{
+ errcontext("during execution of SQL statement: %s", (char *) arg);
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE stmt only */
+ char *relname = NULL; /* For CREATE TABLE stmt only */
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ /*
+ * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, CreateStmt may have been freeed up.
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ RangeVar *rv = cstmt->relation;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+ }
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * OK to analyze, rewrite, and plan this query.
+ *
+ * Switch to appropriate context for constructing query and plan trees
+ * (these can't be in the transaction context, as that will get reset
+ * when the command is COMMIT/ROLLBACK). If we have multiple
+ * parsetrees, we use a separate context for each one, so that we can
+ * free that memory before moving on to the next one. But for the
+ * last (or only) parsetree, just use MessageContext, which will be
+ * reset shortly after completion anyway. In event of an error, the
+ * per_parsetree_context will be deleted when MessageContext is reset.
+ */
+ if (lnext(parsetree_list, parsetree_item) != NULL)
+ {
+ per_parsetree_context =
+ AllocSetContextCreate(MessageContext,
+ "per-parsetree message context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+ }
+ else
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(
+ command,
+ cmdstr,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(
+ querytree_list, cmdstr, 0, NULL);
+
+ /*
+ * Done with the snapshot used for parsing/planning.
+ *
+ * While it looks promising to reuse the same snapshot for query
+ * execution (at least for simple protocol), unfortunately it causes
+ * execution to use a snapshot that has been acquired before locking
+ * any of the tables mentioned in the query. This creates user-
+ * visible anomalies, so refrain. Refer to
+ * https://postgr.es/m/flat/5075D8DF.6050500@fuzzy.cz for details.
+ */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in MessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ cmdstr,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ isTopLevel,
+ true,
+ receiver,
+ receiver,
+ NULL);
+ (*receiver->rDestroy) (receiver);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set
+ * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * new table can be replicated without having to manually run
+ * "alter subscription ... refresh publication"
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+ }
+
+ /*
+ * Restore the GUC variables we set above.
+ */
+ AtEOXact_GUC(true, save_nestlevel);
+
+ /* protect against stack resets during CONCURRENTLY processing */
+ if (error_context_stack == &errcallback)
+ error_context_stack = errcallback.previous;
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2770,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddlmessage(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 42c06af239..a09143dadf 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +260,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +277,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1656,8 +1662,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1703,70 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char * role,
+ const char *search_path, Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+ ListCell *lc;
+
+ /* Reload publications if needed before use. */
+ if (!publications_valid)
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ if (data->publications)
+ list_free_deep(data->publications);
+
+ data->publications = LoadPublications(data->publication_names);
+ MemoryContextSwitchTo(oldctx);
+ publications_valid = true;
+ }
+
+ /* Check if ddl replication is turned on for the publications */
+ foreach(lc, data->publications)
+ {
+ Publication *pub = (Publication *) lfirst(lc);
+ /* TODO need to check relid for table level DDLs */
+ if (!pub->pubactions.pubddl_database && !pub->pubactions.pubddl_table)
+ return;
+ }
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * Output BEGIN if we haven't yet. Avoid for non-transactional
+ * messages.
+ */
+ if (transactional)
+ {
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+ }
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ transactional,
+ prefix,
+ role,
+ search_path,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2052,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl_database =
+ entry->pubactions.pubddl_table = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2112,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl_database = false;
+ entry->pubactions.pubddl_table = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2227,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl_database |= pub->pubactions.pubddl_database;
+ entry->pubactions.pubddl_table |= pub->pubactions.pubddl_table;
/*
* We want to publish the changes as the top-most ancestor
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 849b6d4f7a..ffa0df7728 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1268,13 +1268,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
switch (nodeTag(parsetree))
{
/*
@@ -2104,6 +2097,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8ff3..28ff562d62 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,14 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
+extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
+ const char **role, const char **search_path,
+ bool *transactional, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index cf61fc1e0f..698c5114e6 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -33,7 +33,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index c924ff35f7..6c9b055eb0 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -23,7 +23,7 @@ $node_subscriber->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index 67b4026afa..2902e6fc34 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -32,7 +32,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 1ce696d4a4..6c586877a1 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -19,7 +19,7 @@ $node_subscriber->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/012_collation.pl b/src/test/subscription/t/012_collation.pl
index 2182f7948e..e1b5050663 100644
--- a/src/test/subscription/t/012_collation.pl
+++ b/src/test/subscription/t/012_collation.pl
@@ -76,7 +76,7 @@ $node_subscriber->safe_psql('postgres',
# set up publication, subscription
$node_publisher->safe_psql('postgres',
- q{CREATE PUBLICATION pub1 FOR ALL TABLES});
+ q{CREATE PUBLICATION pub1 FOR ALL TABLES WITH (ddl = '')});
$node_subscriber->safe_psql('postgres',
qq{CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = false)}
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index e7f4a94f19..007a15dfdc 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -25,9 +25,9 @@ $node_subscriber2->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
# publisher
-$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1");
+$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1 WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_all FOR ALL TABLES");
+ "CREATE PUBLICATION pub_all FOR ALL TABLES WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
"CREATE TABLE tab1 (a int PRIMARY KEY, b text) PARTITION BY LIST (a)");
$node_publisher->safe_psql('postgres',
@@ -425,12 +425,12 @@ $node_publisher->safe_psql('postgres',
# and child tables are present but changes will be replicated via the parent's
# identity and only once.
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# for tab4, we publish changes through the "middle" partitioned table
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# prepare data for the initial sync
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
new file mode 100644
index 0000000000..c88c4ea1c0
--- /dev/null
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -0,0 +1,237 @@
+
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $node_subscriber2 = PostgreSQL::Test::Cluster->new('subscriber2');
+$node_subscriber2->init(allows_streaming => 'logical');
+$node_subscriber2->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber2->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+$node_subscriber2->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+# mypub has pubddl_database on
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+# mypub2 has pubddl_database off
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub2 FOR ALL TABLES with (ddl = '');");
+$node_subscriber2->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub2 CONNECTION '$publisher_connstr' PUBLICATION mypub2;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test smae simple CREATE TABLE command is not replicated to subscriber2 (ddl off)
+# Test ALTER TABLE command is replicated on table test_rep
+# Test CREATE INDEX is replicated to subscriber
+# Test CREATE FUNCTION command is replicated to subscriber
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+$node_publisher->safe_psql('postgres', "CREATE INDEX nameindex on test_rep (name)");
+$node_publisher->safe_psql('postgres', qq{CREATE OR REPLACE FUNCTION totalRecords()
+RETURNS integer AS \$total\$
+declare
+ total integer;
+BEGIN
+ SELECT count(*) into total FROM test_rep;
+ RETURN total;
+END;
+\$total\$ LANGUAGE plpgsql;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't1';");
+is($result, qq(0), 'CREATE of t1 is not replicated to subscriber2');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(1), 'CREATE INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT totalRecords();");
+is($result, qq(2), 'CREATE of function totalRecords replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'CREATE FUNCTION totalrecords is not replicated to subscriber2');
+
+# Test ALTER TABLE DROP
+# Test DROP INDEX
+# Test DROP FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->safe_psql('postgres', "DROP INDEX nameindex;");
+$node_publisher->safe_psql('postgres', "DROP FUNCTION totalRecords;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(0), 'DROP INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'DROP FUNCTION totalrecords replicated');
+
+
+# TODO figure out how to set ON_ERROR_STOP = 0 in this test
+# Test failed CREATE/ALTER TABLE on publisher doesn't break replication
+# Table t1 already exits so expect the command to fail
+#$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+#$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (103, 'data103', 1013);");
+
+#$node_publisher->wait_for_catchup('mysub');
+# Verify replication still works
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+#is($result, qq(1), 'DELETE from test_rep replicated');
+
+# Test DDLs inside txn block
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE t2 (a int, b varchar);
+ALTER TABLE test_rep ADD c3 int;
+INSERT INTO test_rep VALUES (3, 'data3', 3);
+COMMIT;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t2;");
+is($result, qq(0), 'CREATE t2 replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep;");
+is($result, qq(2), 'ALTER test_rep ADD replicated');
+
+# Test toggling pubddl_database option off
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = '');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(f|f), 'pubddl_database turned off on mypub');
+$node_publisher->safe_psql('postgres', "CREATE TABLE t3 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't3';");
+is($result, qq(0), 'CREATE t3 is not replicated');
+
+# Test toggling pubddl_database option on
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = 'database');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(t|t), 'pubddl_database turned on on mypub');
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# A somewhat complicated test in plpgsql block with trigger
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE foo (a int);
+CREATE INDEX foo_idx ON foo (a);
+ALTER TABLE foo ADD COLUMN b timestamptz;
+CREATE FUNCTION foo_ts()
+RETURNS trigger AS $$
+BEGIN
+NEW.b := current_timestamp;
+RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
+FOR EACH ROW EXECUTE FUNCTION foo_ts();
+INSERT INTO foo VALUES (1);
+COMMIT;});
+$result = $node_publisher->safe_psql('postgres', "SELECT b from foo where a = 1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result_sub = $node_subscriber->safe_psql('postgres', "SELECT b from foo where a = 1;");
+is($result, qq($result_sub), 'timestamp of insert matches');
+
+# Test CREATE SCHEMA stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE SCHEMA s1");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_namespace WHERE nspname = 's1';");
+is($result, qq(1), 'CREATE SCHEMA s1 is replicated');
+
+# Test CREATE TABLE in new schema s1 followed by insert
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (2, 'b');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t1;");
+is($result, qq(2), 'CREATE TABLE s1.t1 is replicated');
+
+# Test replication works as expected with mismatched search_path on publisher and subscriber
+$node_publisher->append_conf('postgresql.conf', 'search_path = \'s1, public\'');
+$node_publisher->restart;
+# CREATE unqualified table t2, it is s1.t2 under the modified search_path
+$node_publisher->safe_psql('postgres', "CREATE TABLE t2 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (2, 'b');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (3, 'c');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t2;");
+is($result, qq(3), 'CREATE TABLE s1.t2 is replicated');
+
+# Test owner of new table on subscriber matches the owner on publisher
+$node_publisher->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_subscriber->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_publisher->safe_psql('postgres', "SET SESSION AUTHORIZATION 'ddl_replication_user'; CREATE TABLE t5 (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
+is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+
+#TODO TEST certain DDLs are not replicated
+
+pass "DDL replication tests passed!";
+
+$node_subscriber->stop;
+$node_subscriber2->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.32.0
[application/octet-stream] 0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch (4.1K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/5-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch)
download | inline diff:
From e2eb464e82b5e7eb4088c7961b4a1521bd11c30d Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 22 Mar 2022 23:47:52 +0000
Subject: [PATCH 04/10] Enable replication of CREATE MATERIALIZED VIEW AS stmt.
Add tests for both CREATE MATERIALIZED VIEW and CREATE VIEW.
---
src/backend/tcop/utility.c | 42 +++++++++++++++++++++++--
src/test/subscription/t/030_rep_ddls.pl | 18 +++++++++++
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index ffa0df7728..fdda1385c3 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1140,6 +1140,45 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
}
break;
+ /*
+ * CreateTableAsStmt can create either a table a materialized view
+ * and they are handled differently.
+ */
+ case T_CreateTableAsStmt:
+ {
+ CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+ switch(stmt->objtype)
+ {
+ case OBJECT_TABLE:
+ /*
+ * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
+ * 1. A normal CREATE TABLE string that get's logged and replicated via
+ * DDL replication.
+ * 2. Insertions that get replicated by DML replication.
+ */
+ break;
+ case OBJECT_MATVIEW:
+ /*
+ * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ default:
+ break;
+ }
+ break;
+ }
+
/*
* Secondly, commands that may be allowed in Table level DDL replication.
* These are currently handled in the later execution path of the command.
@@ -1155,7 +1194,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/* DropStmt depends on the removeType */
case T_DropStmt:
{
- DropStmt* stmt = (DropStmt*) parsetree;
+ DropStmt *stmt = (DropStmt *) parsetree;
switch (stmt->removeType)
{
/* Maybe allowed in Table level DDL replication, handled in later code path */
@@ -1206,7 +1245,6 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
case T_RuleStmt:
case T_CreateSeqStmt:
case T_AlterSeqStmt:
- case T_CreateTableAsStmt:
case T_RefreshMatViewStmt:
case T_CreatePLangStmt:
case T_CreateConversionStmt:
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index c88c4ea1c0..562efe2cf7 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -226,6 +226,24 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+# Test CREATE MATERIALIZED VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE MATERIALIZED VIEW s1.matview1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+is($result, qq($result_sub), 'CREATE of s1.matview1 is replicated');
+
+# Test CREATE VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.view1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0006-Add-couple-tests-for-DDL-replication-case.patch (4.5K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/6-0006-Add-couple-tests-for-DDL-replication-case.patch)
download | inline diff:
From de7085c2739bc527c566ca208e2b3562a52b3de2 Mon Sep 17 00:00:00 2001
From: Borui Yang <boruiyan@amazon.com>
Date: Mon, 11 Apr 2022 18:02:21 +0000
Subject: [PATCH 06/10] Add couple tests for DDL replication case
---
src/test/subscription/t/030_rep_ddls.pl | 80 +++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 3b15c6d9f0..4e05212068 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -268,6 +268,86 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+# TEST Create DomainStmt
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(t), 'CreateDomain Stmt is replicatted');
+
+# TEST AlterDomainStmt
+$node_publisher->safe_psql('postgres', "Alter domain s1.space_check drop not null;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(f), 'ALTER DOMAIN Stmt is replicated');
+
+#TEST DEFINE Stmt
+$node_publisher->safe_psql('postgres', "CREATE AGGREGATE s1.inc_sum(int) (sfunc = int4pl,stype = int,initcond = 10);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='inc_sum';");
+is($result, qq(1), 'Define stmt is replicated');
+
+#TEST CompositeTypeStmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.compfoo AS (f1 int, f2 text);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='compfoo';");
+is($result, qq(1), 'CompositeType Stmt is replicated');
+
+#TEST CreateEnum Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.mood AS ENUM ('sad', 'ok', 'happy');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='mood';");
+is($result, qq(1), 'CreateEnumType Stmt is replicated');
+
+#TEST AlterEnum Stmt
+$node_publisher->safe_psql('postgres', "ALTER TYPE s1.mood RENAME VALUE 'sad' to 'fine';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_enum e, pg_catalog.pg_type t WHERE e.enumtypid = t.oid AND t.typname='mood' AND e.enumlabel='fine';");
+is($result, qq(1), 'AlterEnumType Stmt is replicated');
+
+#TEST CreateRange Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE floatrange AS RANGE (subtype = float8,subtype_diff = float8mi);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='floatrange';");
+is($result, qq(1), 'CreateRange Stmt is replicated');
+
+#TEST VIEW Stmt
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.vista AS SELECT 'Hello World';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_class c where c.relname='vista';");
+is($result, qq(1), 'VIEW Stmt is replicated');
+
+#TEST CreateFunction Stmt
+$node_publisher->safe_psql('postgres', "CREATE FUNCTION s1.add(a integer, b integer) RETURNS integer LANGUAGE SQL IMMUTABLE RETURN a + b;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='add';");
+is($result, qq(1), 'CreateFunction Stmt is replicated');
+
+#TEST CreateCast Stmt
+$node_publisher->safe_psql('postgres', "CREATE CAST (int AS int4) WITH FUNCTION add(int,int) AS ASSIGNMENT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
+is($result, qq(1), 'CreateCast Stmt is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch (8.3K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/7-0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch)
download | inline diff:
From 0d246b9df312e4a97e417d65af1b9c095c457e8f Mon Sep 17 00:00:00 2001
From: PonyboyYbr <94borelyang@gmail.com>
Date: Wed, 11 May 2022 15:46:40 -0700
Subject: [PATCH 09/10] Support replication of DDL type T_RenameStmt: table
rename is allowed in both database level and table level DDL replication.
Rename of other objects are only allowed in database level DDL replication.
Co-authored-by: Borui Yang <boruiyan@amazon.com>
---
src/backend/commands/alter.c | 5 ++--
src/backend/commands/tablecmds.c | 15 ++++++++++-
src/backend/tcop/utility.c | 20 ++++++++++++---
src/include/commands/alter.h | 3 ++-
src/include/commands/tablecmds.h | 2 +-
src/test/subscription/t/030_rep_ddls.pl | 33 +++++++++++++++++++++++--
6 files changed, 67 insertions(+), 11 deletions(-)
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 5456b8222b..1ca8f99e0c 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -328,7 +328,8 @@ AlterObjectRename_internal(Relation rel, Oid objectId, const char *new_name)
* Return value is the address of the renamed object.
*/
ObjectAddress
-ExecRenameStmt(RenameStmt *stmt)
+ExecRenameStmt(ParseState *pstate, RenameStmt *stmt,
+ bool isCompleteQuery)
{
switch (stmt->renameType)
{
@@ -354,7 +355,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_MATVIEW:
case OBJECT_INDEX:
case OBJECT_FOREIGN_TABLE:
- return RenameRelation(stmt);
+ return RenameRelation(pstate, stmt, isCompleteQuery);
case OBJECT_COLUMN:
case OBJECT_ATTRIBUTE:
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3c9b6409ca..65386f2641 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3820,11 +3820,12 @@ RenameConstraint(RenameStmt *stmt)
* RENAME
*/
ObjectAddress
-RenameRelation(RenameStmt *stmt)
+RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
{
bool is_index_stmt = stmt->renameType == OBJECT_INDEX;
Oid relid;
ObjectAddress address;
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/*
* Grab an exclusive lock on the target table, index, sequence, view,
@@ -3872,6 +3873,18 @@ RenameRelation(RenameStmt *stmt)
is_index_stmt = obj_is_index;
}
+ if (ddlxlog &&
+ ddl_need_xlog(relid, false))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
/* Do the work */
RenameRelationInternal(relid, stmt->newname, false, is_index_stmt);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 2d6b203ac3..a7bfda7b08 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1002,7 +1002,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecRenameStmt(stmt);
+ ExecRenameStmt(pstate, stmt, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1195,8 +1195,20 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
case T_AlterTableStmt:
case T_IndexStmt:
- case T_RenameStmt: /* TODO */
- case T_AlterOwnerStmt: /* TODO */
+ case T_RenameStmt:
+ {
+ RenameStmt *stmt = (RenameStmt *) parsetree;
+ if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ }
+ case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
break;
/* DropStmt depends on the removeType */
@@ -2002,7 +2014,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_RenameStmt:
- address = ExecRenameStmt((RenameStmt *) parsetree);
+ address = ExecRenameStmt(pstate, (RenameStmt *) parsetree, isCompleteQuery);
break;
case T_AlterObjectDependsStmt:
diff --git a/src/include/commands/alter.h b/src/include/commands/alter.h
index 52f5e6f6d2..df9333eb93 100644
--- a/src/include/commands/alter.h
+++ b/src/include/commands/alter.h
@@ -17,9 +17,10 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "utils/relcache.h"
-extern ObjectAddress ExecRenameStmt(RenameStmt *stmt);
+extern ObjectAddress ExecRenameStmt(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern ObjectAddress ExecAlterObjectDependsStmt(AlterObjectDependsStmt *stmt,
ObjectAddress *refAddress);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 24106de2e5..f96bb57d56 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -74,7 +74,7 @@ extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
-extern ObjectAddress RenameRelation(RenameStmt *stmt);
+extern ObjectAddress RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern void RenameRelationInternal(Oid myrelid,
const char *newrelname, bool is_internal,
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index b4df1bfefd..51126b489b 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -30,7 +30,6 @@ $node_subscriber->safe_psql('postgres', $ddl);
$node_subscriber2->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-
# mypub has pubddl_database on
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION mypub FOR ALL TABLES;");
@@ -269,7 +268,7 @@ $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;")
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
# TEST Create DomainStmt
-$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\\s');");
$node_publisher->wait_for_catchup('mysub');
@@ -348,6 +347,36 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST RenameStmt for FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER FUNCTION add RENAME TO plus;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='plus';");
+is($result, qq(1), 'RENAME FUNCTION Stmt is replicated');
+
+#TEST RenameStmt for table
+$node_publisher->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_publisher->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1',
+ "CREATE PUBLICATION mypub3 FOR TABLE t7 with (ddl = 'table');");
+my $publisher_connstr_db1 = $node_publisher->connstr . ' dbname=db1';
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1',
+ "CREATE SUBSCRIPTION mysub3 CONNECTION '$publisher_connstr_db1' PUBLICATION mypub3;"
+);
+$node_publisher->wait_for_catchup('mysub3');
+$node_publisher->safe_psql('db1', "ALTER TABLE t7 RENAME TO newt7;");
+$node_publisher->safe_psql('db1', "ALTER TABLE t8 RENAME TO newt8;");
+$node_publisher->wait_for_catchup('mysub3');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt7';");
+is($result, qq(1), 'Rename t7 to newt7 is replicated');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt8';");
+is($result, qq(0), 'Rename t8 to newt8 is not replicated');
+
#TEST DDL in function
$node_publisher->safe_psql('postgres', qq{
CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
--
2.32.0
[application/octet-stream] 0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch (65.0K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/8-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch)
download | inline diff:
From 5629184c065fb086d6481631bfdd576f8c6e433b Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:47:16 +0000
Subject: [PATCH 01/10] Define DDL replication levels via the CREATE
PUBLICATION command, currently allow Database or Table level DDL replication.
Allow the user to configure either database level or table level
DDL replication via the CREATE PUBLICATION command as in my first
email. Two new columns are added to the pg_publication catalog to
show the DDL replication levels, test output publication.out is
updated accordingly. pg_dump is also modified to accommodate the
pg_publication catalog change.
---
src/backend/catalog/pg_publication.c | 2 +
src/backend/commands/publicationcmds.c | 89 +++++-
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/pg_dump.c | 45 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +-
src/bin/psql/describe.c | 30 +-
src/include/catalog/pg_publication.h | 10 +
src/include/commands/defrem.h | 1 +
src/test/regress/expected/publication.out | 372 +++++++++++-----------
10 files changed, 364 insertions(+), 199 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index e2c8bcb279..71a259153c 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1006,6 +1006,8 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubactions.pubddl_database = pubform->pubddl_database;
+ pub->pubactions.pubddl_table = pubform->pubddl_table;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e645741e4..edceaa0c4e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -80,15 +80,18 @@ static void PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok);
static void
parse_publication_options(ParseState *pstate,
List *options,
+ bool for_all_tables,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *ddl_level_given)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *ddl_level_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -96,6 +99,16 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ if (for_all_tables)
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ else
+ {
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+ }
/* Parse options */
foreach(lc, options)
@@ -154,6 +167,55 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "ddl") == 0)
+ {
+ char *ddl_level;
+ List *ddl_list;
+ ListCell *lc;
+
+ if (*ddl_level_given)
+ errorConflictingDefElem(defel, pstate);
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
+ ddl_level = defGetString(defel);
+
+ if (!SplitIdentifierString(ddl_level, ',', &ddl_list))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax for \"ddl\" option")));
+
+ /* Process the option list. */
+ foreach(lc, ddl_list)
+ {
+ char *publish_opt = (char *) lfirst(lc);
+
+ if (strcmp(publish_opt, "database") == 0)
+ {
+ if (!for_all_tables)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("DDL replication on the database level is only supported in FOR ALL TABLES publication")));
+ else
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ }
+ else if (strcmp(publish_opt, "table") == 0)
+ pubactions->pubddl_table = true;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized \"ddl\" value: \"%s\"", publish_opt)));
+ }
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -761,6 +823,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Datum values[Natts_pg_publication];
HeapTuple tup;
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -803,9 +866,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
parse_publication_options(pstate,
stmt->options,
+ stmt->for_all_tables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -822,6 +887,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubddl_database - 1] =
+ BoolGetDatum(pubactions.pubddl_database);
+ values[Anum_pg_publication_pubddl_table - 1] =
+ BoolGetDatum(pubactions.pubddl_table);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -908,6 +977,7 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
bool replaces[Natts_pg_publication];
Datum values[Natts_pg_publication];
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -916,11 +986,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
List *root_relids = NIL;
ListCell *lc;
+ pubform = (Form_pg_publication) GETSTRUCT(tup);
+
parse_publication_options(pstate,
stmt->options,
+ pubform->puballtables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -1024,6 +1098,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
+ if (ddl_level_given)
+ {
+ values[Anum_pg_publication_pubddl_database - 1] = BoolGetDatum(pubactions.pubddl_database);
+ replaces[Anum_pg_publication_pubddl_database - 1] = true;
+
+ values[Anum_pg_publication_pubddl_table - 1] = BoolGetDatum(pubactions.pubddl_table);
+ replaces[Anum_pg_publication_pubddl_table - 1] = true;
+ }
+
if (publish_via_partition_root_given)
{
values[Anum_pg_publication_pubviaroot - 1] = BoolGetDatum(publish_via_partition_root);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9e8b..dd235d66d0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl_database |= pubform->pubddl_database;
+ pubdesc->pubactions.pubddl_table |= pubform->pubddl_table;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72e49..8860510ac1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3868,6 +3868,8 @@ getPublications(Archive *fout, int *numPublications)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubddl_database;
+ int i_pubddl_table;
int i,
ntups;
@@ -3882,23 +3884,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubddl_database, p.pubddl_table "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3915,6 +3923,8 @@ getPublications(Archive *fout, int *numPublications)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubddl_database = PQfnumber(res, "pubddl_database");
+ i_pubddl_table = PQfnumber(res, "pubddl_table");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3939,6 +3949,10 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubddl_database =
+ (strcmp(PQgetvalue(res, i, i_pubddl_database), "t") == 0);
+ pubinfo[i].pubddl_table =
+ (strcmp(PQgetvalue(res, i, i_pubddl_table), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4018,6 +4032,29 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
appendPQExpBufferStr(query, "'");
+ appendPQExpBufferStr(query, ", ddl = '");
+ first = true;
+
+ if (pubinfo->pubddl_database)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "database");
+ first = false;
+ }
+
+ if (pubinfo->pubddl_table)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "table");
+ first = false;
+ }
+
+ appendPQExpBufferStr(query, "'");
+
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..8c5ef01cb0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,8 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1f08716f69..cbfcabcfa8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2420,7 +2420,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2429,9 +2429,9 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub2
FOR ALL TABLES
- WITH (publish = \'\');',
+ WITH (publish = \'\', ddl = \'\');',
regexp => qr/^
- \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '');\E
+ \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2440,7 +2440,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub3;',
regexp => qr/^
- \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2449,7 +2449,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub4;',
regexp => qr/^
- \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924a23..b07c6b1841 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,15 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_database AS \"%s\"",
+ gettext_noop("Database level DDLs"));
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_table AS \"%s\"",
+ gettext_noop("Table level DDLs"));
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6181,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6198,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6212,11 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ {
+ appendPQExpBufferStr(&buf,
+ ", pubddl_database, pubddl_table");
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6265,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols += 2;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6281,11 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ {
+ printTableAddHeader(&cont, gettext_noop("Database level DDL"), true, align);
+ printTableAddHeader(&cont, gettext_noop("Table level DDL"), true, align);
+ }
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6296,11 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ {
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
+ printTableAddCell(&cont, PQgetvalue(res, i, 10), false, false);
+ }
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba429..a94e152256 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -20,6 +20,7 @@
#include "catalog/genbki.h"
#include "catalog/objectaddress.h"
#include "catalog/pg_publication_d.h"
+#include "tcop/utility.h"
/* ----------------
* pg_publication definition. cpp turns this into
@@ -54,6 +55,12 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if database level ddls are published */
+ bool pubddl_database;
+
+ /* true if table level ddls are published */
+ bool pubddl_table;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +79,8 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationActions;
typedef struct PublicationDesc
@@ -157,5 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..073f46ef1a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -15,6 +15,7 @@
#define DEFREM_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_publication.h"
#include "nodes/params.h"
#include "parser/parse_node.h"
#include "tcop/dest.h"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 398c0f38f6..180ebd359e 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | t | t | t | f | f | f | t | t
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.32.0
[application/octet-stream] 0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch (17.5K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/9-0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch)
download | inline diff:
From 4040dc4378ec520b9641557ea03af3401be403d1 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 15 Apr 2022 02:02:10 +0000
Subject: [PATCH 07/10] Enable logging and replication of DDLs executed inside
function and procedures. Also fixed a bug where DDL with SUBCOMMAND gets
logged twice.
---
.../test_decoding/expected/ddlmessages.out | 47 +++++++++++++++++--
contrib/test_decoding/sql/ddlmessages.sql | 19 +++++++-
src/backend/catalog/pg_publication.c | 5 +-
src/backend/commands/tablecmds.c | 6 +--
src/backend/tcop/utility.c | 26 +++++-----
src/include/catalog/pg_publication.h | 2 +-
src/include/commands/tablecmds.h | 3 +-
src/test/subscription/t/030_rep_ddls.pl | 38 +++++++++++++++
8 files changed, 121 insertions(+), 25 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 0376f36c24..823029d03d 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -11,7 +11,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d
init
(1 row)
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -24,9 +24,9 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
@@ -34,6 +34,44 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+SELECT func_ddl ('tab_from_func');
+ func_ddl
+----------
+
+(1 row)
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
+ LANGUAGE plpgsql;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ BEGIN
+ table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
+ table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
+ COMMIT
+(7 rows)
+
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
@@ -41,4 +79,5 @@ SELECT pg_drop_replication_slot('regression_slot');
(1 row)
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index c23610f9b4..3082671488 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -9,7 +9,7 @@ SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -25,7 +25,24 @@ ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+
+SELECT func_ddl ('tab_from_func');
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 92d2763e83..b6fb0d2c35 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1161,15 +1161,12 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
* Checks if DDL on relation (relid) need xlog for logical replication
*/
bool
-ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly)
{
List *allTablePubs = NIL;
List *tablePubs = NIL;
ListCell *lc;
- /* Only replicate toplevel DDL command */
- if (!isTopLevel)
- return false;
if (relid == InvalidOid && !forAllTabPubOnly)
return false;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6d1487951f..3c9b6409ca 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1333,14 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
- bool ddlxlog = XLogLogicalInfoActive();
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1466,7 +1466,7 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
/* DROP RELATION or INDEX are allowed in table level DDL replication */
if (tableOid != InvalidOid &&
- !ddl_need_xlog(tableOid, false, isTopLevel))
+ !ddl_need_xlog(tableOid, false))
ddlxlog = false;
}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 34c7db8cc9..2d6b203ac3 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -87,7 +87,8 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -988,7 +989,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(pstate, stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1128,7 +1129,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1170,7 +1171,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1221,7 +1222,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1318,7 +1319,7 @@ ProcessUtilitySlow(ParseState *pstate,
* Consider logging the DDL command if logical logging is enabled and this is
* a complete top level query.
*/
- if (XLogLogicalInfoActive() && isTopLevel)
+ if (XLogLogicalInfoActive())
LogLogicalDDLCommand(parsetree, queryString);
}
@@ -1530,7 +1531,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1766,7 +1768,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on.
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1993,7 +1996,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel, isCompleteQuery);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -2214,7 +2217,8 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery)
{
switch (stmt->removeType)
{
@@ -2229,7 +2233,7 @@ ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(pstate, stmt, isTopLevel);
+ RemoveRelations(pstate, stmt, isCompleteQuery);
break;
default:
RemoveObjects(stmt);
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index a94e152256..8c114b2447 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -166,6 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
-extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 68781365de..24106de2e5 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -28,7 +28,8 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop,
+ bool isCompleteQuery);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 4e05212068..34b9d51eb1 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -348,6 +348,44 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST DDL in function
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+\$\$
+LANGUAGE plpgsql;});
+
+$node_publisher->safe_psql('postgres', "SELECT func_ddl('func_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.func_table where c3 = 22;");
+is($result, qq(1), 'DDLs in function are replicated');
+
+#TEST DDL in procedure
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE procedure proc_ddl (tname varchar(20))
+LANGUAGE plpgsql AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END \$\$;});
+
+$node_publisher->safe_psql('postgres', "CALL proc_ddl('proc_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
+is($result, qq(1), 'DDLs in procedure are replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch (4.8K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/10-0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch)
download | inline diff:
From c28b8bb5aaf2a9d39ef05755f4449a4f8c6fb938 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 29 Apr 2022 16:57:03 +0000
Subject: [PATCH 08/10] Fail replication worker on DDL command that rewrites
table using volatile functions, such as ALTER TABLE tab ADD COLUMN col
DEFAULT volatile_expr. This is to avoid data mismatch compared to the
publisher. We can potentially unblock this type of command when table rewrite
is supported in logical replication.
---
src/backend/replication/logical/worker.c | 44 ++++++++++++++++++++++++
src/test/subscription/t/030_rep_ddls.pl | 22 ++++++++++++
2 files changed, 66 insertions(+)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 905f47faf5..01db102e3c 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -157,6 +157,7 @@
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
+#include "parser/parse_expr.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -2547,6 +2548,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
commandTag = CreateCommandTag((Node *)command);
+ /* The following DDL commands need special handling */
+
/*
* Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
@@ -2594,6 +2597,47 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
sstmt->intoClause->skipData = true;
}
}
+ /*
+ * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
+ * Until we support logical replication of table rewrite, see ATRewriteTables()
+ * for details on table rewrite.
+ */
+ else if (commandTag == CMDTAG_ALTER_TABLE)
+ {
+ AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
+ ListCell *lc;
+
+ foreach(lc, atstmt->cmds)
+ {
+ AlterTableCmd *cmd = lfirst_node(AlterTableCmd, lc);
+
+ if (cmd->subtype == AT_AddColumn)
+ {
+ ColumnDef *colDef;
+ ListCell *c;
+
+ colDef = castNode(ColumnDef, cmd->def);
+ foreach(c, colDef->constraints)
+ {
+ Constraint *con = lfirst_node(Constraint, c);
+
+ if (con->contype == CONSTR_DEFAULT)
+ {
+ Node *expr;
+ ParseState *pstate = make_parsestate(NULL);
+
+ expr = transformExpr(pstate, copyObject(con->raw_expr), EXPR_KIND_COLUMN_DEFAULT);
+ if (contain_volatile_functions(expr))
+ {
+ elog(ERROR,
+ "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
+ cmdstr);
+ }
+ }
+ }
+ }
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 34b9d51eb1..b4df1bfefd 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -386,7 +386,29 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
is($result, qq(1), 'DDLs in procedure are replicated');
+# Test Alter table alter column type stmt is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN name type TEXT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT data_type FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'name';");
+is($result, qq(text), 'Alter table column type stmt is replicated');
+
+# Test Alter table add column default 0.01 is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN non_volatile double precision DEFAULT 0.01;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
+is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+
#TODO TEST certain DDLs are not replicated
+# Test DDL statement that rewrites table with volatile functions are not replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'volatile';");
+is($result, qq(1), 'Alter table add column default random() is executed on the publisher DB.');
+
+$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0010-Handle-partitioned-table-creation-on-the-apply-worke.patch (11.5K, ../../CAAD30U+CRgUgkAg33KzNBKwCbsgiSc5z3NYvxNzEfS0Zg2S1WA@mail.gmail.com/11-0010-Handle-partitioned-table-creation-on-the-apply-worke.patch)
download | inline diff:
From 8f4af5ac2a5f35ba2b85dfa525e3d836cfa570b0 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 13 May 2022 19:47:44 +0000
Subject: [PATCH 10/10] Handle partitioned table creation on the apply worker:
whether a partitioned table should be added to pg_subscription_rel catalog
depends on the setting of publish_via_partition_root of the publication. Thus
we need to connect to the source DB and check whehter the partitioned table
should be subscribed.
---
src/backend/commands/subscriptioncmds.c | 54 +++++++++++++++++++++-
src/backend/replication/logical/worker.c | 35 ++++++++++++---
src/include/commands/subscriptioncmds.h | 3 ++
src/test/regress/expected/psql.out | 6 +--
src/test/subscription/t/030_rep_ddls.pl | 57 ++++++++++++++++++++++++
5 files changed, 145 insertions(+), 10 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 690cdaa426..b6c2fd1004 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -23,7 +23,6 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/objectaddress.h"
-#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
@@ -746,6 +745,59 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
return myself;
}
+/*
+ * Check if a partitioned table is being published by any publication subscirbed by the subscription.
+ * Whether a partitioned table is published also depends on the publication option
+ * publish_via_partition_root. But the subscriber doesn't know the setting of publish_via_partition_root,
+ * this is why we need to check the source DB so that we can decide whether to subscribe to the partitioned
+ * table (could be either root or leaf table) during replication of create partitioned table.
+ */
+bool
+IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name)
+{
+ char *err;
+ WalRcvExecResult *res;
+ StringInfoData cmd;
+ Oid tableRow[1] = {BOOLOID};
+ bool result;
+
+ WalReceiverConn *wrconn;
+ /* Load the library providing us libpq calls. */
+ load_file("libpqwalreceiver", false);
+
+ /* Try to connect to the publisher. */
+ wrconn = walrcv_connect(sub->conninfo, true, sub->name, &err);
+ if (!wrconn)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not connect to the publisher: %s", err)));
+
+ PG_TRY();
+ {
+ initStringInfo(&cmd);
+ appendStringInfoString(&cmd, "SELECT TRUE\n"
+ " FROM pg_catalog.pg_publication_tables t\n"
+ " WHERE t.pubname IN (");
+ get_publications_str(sub->publications, &cmd, true);
+ appendStringInfoString(&cmd, ") AND t.schemaname = '");
+ appendStringInfoString(&cmd, schema_name);
+ appendStringInfoString(&cmd, "' AND t.tablename = '");
+ appendStringInfoString(&cmd, table_name);
+ appendStringInfoString(&cmd, "'");
+
+ res = walrcv_exec(wrconn, cmd.data, 1, tableRow);
+ pfree(cmd.data);
+ result = tuplestore_tuple_count(res->tuplestore) > 0;
+ }
+ PG_FINALLY();
+ {
+ walrcv_disconnect(wrconn);
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
static void
AlterSubscription_refresh(Subscription *sub, bool copy_data,
List *validate_publications)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 01db102e3c..610147ca77 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -143,6 +143,7 @@
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_tablespace.h"
+#include "commands/subscriptioncmds.h"
#include "commands/tablecmds.h"
#include "commands/tablespace.h"
#include "commands/trigger.h"
@@ -2545,6 +2546,7 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
bool snapshot_set = false;
char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ bool is_partitioned_table = false;
commandTag = CreateCommandTag((Node *)command);
@@ -2561,6 +2563,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
+ if (cstmt->inhRelations != NIL || cstmt->partspec != NULL)
+ is_partitioned_table = true;
}
else if (commandTag == CMDTAG_CREATE_TABLE_AS)
{
@@ -2762,12 +2766,31 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
if (relid != InvalidOid)
{
- AddSubscriptionRelState(MySubscription->oid, relid,
- SUBREL_STATE_INIT,
- InvalidXLogRecPtr);
- ereport(DEBUG1,
- (errmsg_internal("table \"%s\" added to subscription \"%s\"",
- relname, MySubscription->name)));
+ bool subscribe_table = true;
+
+ if (is_partitioned_table)
+ {
+ Relation rel = RelationIdGetRelation(relid);
+ char *table_name = RelationGetRelationName(rel);
+ char *schema_name = get_namespace_name(RelationGetNamespace(rel));
+ /*
+ * Connect to the source DB and check whehter the partitioned table should be subscribed.
+ * Because it depends on the setting of publish_via_partition_root, which the subscription
+ * doesn't know.
+ */
+ subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
+ RelationClose(rel);
+ }
+
+ if (subscribe_table)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
}
}
}
diff --git a/src/include/commands/subscriptioncmds.h b/src/include/commands/subscriptioncmds.h
index 2cbe7d7b65..451953af60 100644
--- a/src/include/commands/subscriptioncmds.h
+++ b/src/include/commands/subscriptioncmds.h
@@ -16,6 +16,7 @@
#define SUBSCRIPTIONCMDS_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_subscription.h"
#include "parser/parse_node.h"
extern ObjectAddress CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
@@ -26,4 +27,6 @@ extern void DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel);
extern ObjectAddress AlterSubscriptionOwner(const char *name, Oid newOwnerId);
extern void AlterSubscriptionOwner_oid(Oid subid, Oid newOwnerId);
+extern bool IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name);
+
#endif /* SUBSCRIPTIONCMDS_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2a38a93a3b..7c497605ed 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 51126b489b..a03d598266 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -431,6 +431,63 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+# Test partitioned table creation is replicated based on the setting of publish_via_partition_root
+$node_publisher->safe_psql('postgres', qq(
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table is created and populated');
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table and data is replicated');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(0), 'Root table of the partitioned table is not subscribed since publish_via_partition_root is false by default');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(2), 'Only leaf tables of the partitioned table are subscribed since publish_via_partition_root is false by default');
+
+# Test only root partition table is subscribed when publish_via_partition_root is enabled
+$node_publisher->safe_psql('postgres', qq(
+ DROP TABLE s1.test_part_a;
+ ALTER PUBLICATION mypub SET (publish_via_partition_root = true);
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(1), 'Only root table of the partitioned table is subscribed since publish_via_partition_root is enabled');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(0), 'Leaf tables of the partitioned table are not subscribed since publish_via_partition_root is enabled');
+
#TODO TEST certain DDLs are not replicated
# Test DDL statement that rewrites table with volatile functions are not replicated
$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-23 11:54 Amit Kapila <amit.kapila16@gmail.com>
parent: Euler Taveira <euler@eulerto.com>
2 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-05-23 11:54 UTC (permalink / raw)
To: Euler Taveira <euler@eulerto.com>; +Cc: Dilip Kumar <dilipbalaut@gmail.com>; Zheng Li <zhengli10@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
On Thu, Apr 14, 2022 at 7:45 PM Euler Taveira <euler@eulerto.com> wrote:
>
> You should forbid it. Unless you can decompose the command into multiple SQL
> commands to make it a safe operation for logical replication.
>
> Let's say you want to add a column with a volatile default.
>
> ALTER TABLE foo ADD COLUMN bar double precision DEFAULT random();
>
> If you replicate the DDL command as is, you will have different data
> downstream. You should forbid it. However, this operation can be supported if
> the DDL command is decomposed in multiple steps.
>
> -- add a new column without DEFAULT to avoid rewrite
> ALTER TABLE foo ADD COLUMN bar double precision;
>
> -- future rows could use the DEFAULT expression
> -- it also doesn't rewrite the table
> ALTER TABLE foo ALTER COLUMN bar SET DEFAULT random();
>
> -- it effectively rewrites the table
> -- all rows are built from one source node
> -- data will be the same on all nodes
> UPDATE foo SET bar = random();
>
While thinking about this, I see more to it than this. Say, we are
able to decompose/split the DDL command with the help of deparser, do
we want to update the additional rows on the subscriber that didn't
exist on the publisher? For example,
A table on the publisher side has rows:
ddl_test(foo)
a
----
1
2
The same table on the subscriber side has rows:
ddl_test(foo)
a
----
1
2
3
4
Now, say, the user has added a bar column with "ALTER TABLE foo ADD
COLUMN bar double precision NOT NULL DEFAULT random();" If we compare
with replication of DMLs like (UPDATE ddl_test SET bar = random();),
the replication won't update rows with values (3 and 4) on subscriber
as they don't exist on the publisher. However, if we follow the same
here for DDL replication of Alter, it will fail because of NOT NULL
constraint. So, it seems we should update all the existing rows on the
subscriber to make replication of such constraints successful. It
seems that IBM's replication solution allows replication of such DDLs
and does update all existing rows on the target table [1][2].
I think it would be tricky to update the rows in subscriber that
doesn't exist in the publisher as we need to distinguish/find such
rows during apply but I think we can find some way to achieve this if
we decide to go this way.
We can also conclude that we want to restrict the replication of Alter
Table for such cases but as other databases seem to support this, I
think it is worth trying to support such an operation. If it turns out
to be too complex or not at all feasible then we can always exclude it
from the first version.
Thoughts?
[1] - https://www.ibm.com/docs/en/idr/10.2.1?topic=replication-adding-existing-columns-subscription-unidir...
(... Columns are added to the target table with the same data type,
null characteristic, and default value as the matching columns in the
source table... Rows that existed in the target table before the new
column is added will have a NULL or default value for the new column.
[2] - https://www.ibm.com/docs/en/idr/11.4.0?topic=replication-alter-add-column-command-multidirectional
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-23 15:40 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-05-23 15:40 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Euler Taveira <euler@eulerto.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh singarapu <rajesh.rs0541@gmail.com>
> Now, say, the user has added a bar column with "ALTER TABLE foo ADD
> COLUMN bar double precision NOT NULL DEFAULT random();" If we compare
> with replication of DMLs like (UPDATE ddl_test SET bar = random();),
> the replication won't update rows with values (3 and 4) on subscriber
> as they don't exist on the publisher. However, if we follow the same
> here for DDL replication of Alter, it will fail because of NOT NULL
> constraint. So, it seems we should update all the existing rows on the
> subscriber to make replication of such constraints successful. It
> seems that IBM's replication solution allows replication of such DDLs
> and does update all existing rows on the target table [1][2].
>
> I think it would be tricky to update the rows in subscriber that
> doesn't exist in the publisher as we need to distinguish/find such
> rows during apply but I think we can find some way to achieve this if
> we decide to go this way.
I think the behavior that makes most sense here is to replicate the default
values of the new column for existing rows from the publisher, and generate
default values for the additional rows on the subscriber.
> We can also conclude that we want to restrict the replication of Alter
> Table for such cases but as other databases seem to support this, I
> think it is worth trying to support such an operation. If it turns out
> to be too complex or not at all feasible then we can always exclude it
> from the first version.
I think it might be OK that we exclude such cases (fail apply worker on
occurence) for the first version. I've spent some time on it and it seems
to be a high effort, low reward task at the moment.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-26 05:45 Ajin Cherian <itsajin@gmail.com>
parent: Ajin Cherian <itsajin@gmail.com>
2 siblings, 1 reply; 449+ messages in thread
From: Ajin Cherian @ 2022-05-26 05:45 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com, Hou, Zhijie/侯 志杰 <houzj.fnst@fujitsu.com>
On Tue, May 10, 2022 at 9:33 PM Ajin Cherian <itsajin@gmail.com> wrote:
>
> On Fri, May 6, 2022 at 11:24 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > As we have hacked CreatePublication function for this POC, the
> > regression tests are not passing but we can easily change it so that
> > we invoke new functionality with the syntax proposed in this thread or
> > with some other syntax and we shall do that in the next patch unless
> > this approach is not worth pursuing.
> >
> > This POC is prepared by Ajin Cherian, Hou-San, and me.
> >
> > Thoughts?
> >
> > [1] - https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.no-ip.org
>
> I have updated Amit's patch by including a public action "create" when
> creating publication which is turned off by default.
> Now the 'make check' tests pass. I also fixed a problem that failed to
> create tables when the table has a primary key.
I have updated this "deparse" patch-set to include support for 'drop table':
Only the drop command of those tables in the publication is replicated.
This is achieved with two event trigger handlers, 'command start' and
'command end' for drop table commands.
The 'command start' event handler logs a ddl message with the relids
of the tables that are dropped which the output plugin (pgoutput)
stores in its internal data structure after verifying that it is for a
table that is part of the publication. Later the 'command end' event
handler
sends the actual drop message. Pgoutput on receiving the command end,
only sends out the drop command only if it is for one of the
relids marked for deleting. The reason we have to do this is because,
once the logical decoder receives the 'command end' message,
the relid of the table is no longer valid as it has been deleted as
part of invalidations received for the drop table command.
It is no longer possible to verify if the table is part of the
publication list or not. To make this possible, I have added two more
elements
to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well,
but that would mean sending spurious ddl messages for tables that
are not part of the publication.
Example:
publisher:
postgres=# create table test (a int, b text);
CREATE TABLE
postgres=# create table test1 (a int, b text);
CREATE TABLE
postgres=# create table test2 (a int, b text);
CREATE TABLE
postgres=# create publication tap_pub for TABLE test, test1 with
(publish='ddl'); CREATE PUBLICATION
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+-------+-------+-------
public | test | table | ajin
public | test1 | table | ajin
public | test2 | table | ajin
(3 rows)
subscriber:
postgres=# create table test (a int, b text);
CREATE TABLE
postgres=# create table test1 (a int, b text);
CREATE TABLE
postgres=# create table test2 (a int, b text);
CREATE TABLE
postgres=# CREATE SUBSCRIPTION sub CONNECTION '<CONNINFO>' PUBLICATION tap_pub;
NOTICE: created replication slot "sub" on publisher
CREATE SUBSCRIPTION
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+-------+-------+-------
public | test | table | ajin
public | test1 | table | ajin
public | test2 | table | ajin
(3 rows)
publisher:
postgres=# drop table test1;
DROP TABLE
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+-------+-------+-------
public | test | table | ajin
public | test2 | table | ajin
(2 rows)
subscriber:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+-------+-------+-------
public | test | table | ajin
public | test2 | table | ajin
(2 rows)
>>>> DROP table replicated.
publisher:
postgres=# drop table test2;
DROP TABLE
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+------+-------+-------
public | test | table | ajin
(1 row)
subscriber:
postgres=# \d
List of relations
Schema | Name | Type | Owner
--------+-------+-------+-------
public | test | table | ajin
public | test2 | table | ajin
(2 rows)
>>>DROP table not replicated since table not in publication list.
regards,
Ajin Cherian
Attachments:
[application/octet-stream] v5-0001-Functions-to-deparse-DDL-commands.patch (63.9K, ../../CAFPTHDaBodoZ5c7U1uyokbvq+zUvhJ4ps-7H66nHGw45UnO0OQ@mail.gmail.com/2-v5-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 57bf2fa4be0041e9b10f3c1113a4560527447ed4 Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 25 May 2022 04:04:16 -0400
Subject: [PATCH v5 1/3] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE specific functions. We can extend
it as we need more functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 1487 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 6 +
src/include/tcop/ddl_deparse.h | 9 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2263 insertions(+)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..ac92f10
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,1487 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..c65bcf3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,11 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..e979ee5
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,9 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
1.8.3.1
[application/octet-stream] v5-0003-Add-support-for-drop-tables-in-logical-replicatio.patch (75.9K, ../../CAFPTHDaBodoZ5c7U1uyokbvq+zUvhJ4ps-7H66nHGw45UnO0OQ@mail.gmail.com/3-v5-0003-Add-support-for-drop-tables-in-logical-replicatio.patch)
download | inline diff:
From 3b150486c25f9e6668e4f5054d2ba263c349badb Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 25 May 2022 05:37:53 -0400
Subject: [PATCH v5 3/3] Add support for drop tables in logical replication.
Change the pubaction from 'create' to 'ddl'. Also add elements relid
and cmdtype to the ddl message xlog and logical message.
---
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/ddl_deparse.c | 20 ++
src/backend/commands/event_trigger.c | 68 +++++-
src/backend/commands/publicationcmds.c | 79 ++++---
src/backend/replication/logical/ddlmessage.c | 4 +-
src/backend/replication/logical/logical.c | 17 +-
src/backend/replication/logical/reorderbuffer.c | 21 +-
src/backend/replication/pgoutput/pgoutput.c | 45 +++-
src/backend/utils/cache/relcache.c | 2 +-
src/bin/pg_dump/pg_dump.c | 20 +-
src/bin/pg_dump/pg_dump.h | 2 +-
src/bin/psql/describe.c | 20 +-
src/include/catalog/pg_proc.dat | 3 +
src/include/catalog/pg_publication.h | 4 +-
src/include/replication/ddlmessage.h | 14 +-
src/include/replication/output_plugin.h | 4 +
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 7 +-
src/include/tcop/ddl_deparse.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 288 ++++++++++++------------
21 files changed, 405 insertions(+), 222 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index ac92f10..bdf4ad3 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -1399,6 +1399,26 @@ deparse_simple_command(CollectedCommand *cmd)
}
char *
+deparse_drop_table(const char *objidentity)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
deparse_utility_command(CollectedCommand *cmd)
{
OverrideSearchPath *overridePath;
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 9c669c4..983f131 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -2190,9 +2190,50 @@ stringify_adefprivs_objtype(ObjectType objtype)
* Deparse the ddl command and log it.
*/
Datum
+publication_ddl_deparse_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_DropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
publication_ddl_deparse(PG_FUNCTION_ARGS)
{
ListCell *lc;
+ slist_iter iter;
if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
elog(ERROR, "not fired by event trigger manager");
@@ -2201,27 +2242,36 @@ publication_ddl_deparse(PG_FUNCTION_ARGS)
{
CollectedCommand *cmd = lfirst(lc);
char *json_string;
- static int i;
if (cmd->type == SCT_Simple &&
!OidIsValid(cmd->d.simple.address.objectId))
continue;
- /*
- * Fixme: This is just for the development purpose and needs to be
- * removed later.
- */
- elog(LOG, "DDL command no. %d", ++i);
-
/* Deparse the DDL command and WAL log it to allow decoding of the same. */
json_string = deparse_utility_command(cmd);
if (json_string == NULL)
continue;
-
- LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, cmd->type,
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_Create,
json_string, strlen(json_string) + 1);
}
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ SQLDropObject *obj;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ char *command;
+
+ command = deparse_drop_table(obj->objidentity);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_DropEnd,
+ command, strlen(command) + 1);
+ }
+ }
+
return PointerGetDatum(NULL);
}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index fccd531..48a5601 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -97,7 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
- pubactions->pubcreate = false;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -144,8 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
- else if (strcmp(publish_opt, "create") == 0)
- pubactions->pubcreate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -761,7 +761,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
- ObjectAddress referenced;
+ ObjectAddress referenced_start, referenced_end;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -826,8 +826,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
- values[Anum_pg_publication_pubcreate - 1] =
- BoolGetDatum(pubactions.pubcreate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -900,33 +900,60 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
* We need to enhance this once the approach for DDL replication is
* finalized.
*/
- if (stmt->for_all_tables && pubactions.pubcreate)
+ if (pubactions.pubddl)
{
- CreateEventTrigStmt *ddl_trigg;
- Node *args = NULL;
- List *tags = NIL;
- Oid event_trig_id;
- char trigger_name[NAMEDATALEN];
+ CreateEventTrigStmt *ddl_trigg_start, *ddl_trigg_end;
+ Node *end_arg1 = NULL;
+ Node *start_arg1 = NULL;
+ Node *arg2 = NULL;
+ List *end_tags = NIL;
+ List *start_tags = NIL;
+ Oid event_trig_start_id, event_trig_end_id;
+ char trigger_name_start[NAMEDATALEN];
+ char trigger_name_end[NAMEDATALEN];
+
+ ddl_trigg_end = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_end, sizeof(trigger_name_end), "pg_deparse_trig_end_%u",
+ puboid);
+ ddl_trigg_end->trigname = pstrdup(trigger_name_end);
+ ddl_trigg_end->eventname = "ddl_command_end";
+ ddl_trigg_end->funcname = SystemFuncName("publication_ddl_deparse");
+
+ end_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ end_tags = list_make1(end_arg1);
+
+ /* support create table only when publication is for all tables */
+ if (stmt->for_all_tables)
+ {
+ arg2 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ end_tags = lappend(end_tags, arg2);
+ }
+ ddl_trigg_end->whenclause = list_make1(makeDefElem("tag", (Node *) end_tags, -1));
+ event_trig_end_id = CreateEventTrigger(ddl_trigg_end);
- ddl_trigg = makeNode(CreateEventTrigStmt);
+ ddl_trigg_start = makeNode(CreateEventTrigStmt);
- snprintf(trigger_name, sizeof(trigger_name), "pg_deparse_trig_%u",
+ snprintf(trigger_name_start, sizeof(trigger_name_start), "pg_deparse_trig_start_%u",
puboid);
- ddl_trigg->trigname = pstrdup(trigger_name);
- ddl_trigg->eventname = "ddl_command_end";
- ddl_trigg->funcname = SystemFuncName("publication_ddl_deparse");
+ ddl_trigg_start->trigname = pstrdup(trigger_name_start);
+ ddl_trigg_start->eventname = "ddl_command_start";
+ ddl_trigg_start->funcname = SystemFuncName("publication_ddl_deparse_start");
- args = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
- tags = list_make1(args);
- ddl_trigg->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
- event_trig_id = CreateEventTrigger(ddl_trigg);
+ start_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ start_tags = list_make1(start_arg1);
+ ddl_trigg_start->whenclause = list_make1(makeDefElem("tag", (Node *) start_tags, -1));
+ event_trig_start_id = CreateEventTrigger(ddl_trigg_start);
/*
- * Register the event trigger as internally dependent on the
+ * Register the event triggers as internally dependent on the
* publication.
*/
- ObjectAddressSet(referenced, EventTriggerRelationId, event_trig_id);
- recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
+ ObjectAddressSet(referenced_end, EventTriggerRelationId, event_trig_end_id);
+ recordDependencyOn(&referenced_end, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_start, EventTriggerRelationId, event_trig_start_id);
+ recordDependencyOn(&referenced_start, &myself, DEPENDENCY_INTERNAL);
}
table_close(rel, RowExclusiveLock);
@@ -1070,8 +1097,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
- values[Anum_pg_publication_pubcreate - 1] = BoolGetDatum(pubactions.pubcreate);
- replaces[Anum_pg_publication_pubcreate - 1] = true;
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
index c40cd62..5093523 100644
--- a/src/backend/replication/logical/ddlmessage.c
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -42,7 +42,7 @@
* Write logical decoding DDL message into XLog.
*/
XLogRecPtr
-LogLogicalDDLMessage(const char *prefix, Oid relid, NodeTag cmdtype,
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
const char *message, size_t size)
{
xl_logical_ddl_message xlrec;
@@ -57,6 +57,8 @@ LogLogicalDDLMessage(const char *prefix, Oid relid, NodeTag cmdtype,
/* trailing zero is critical; see logicalddlmsg_desc */
xlrec.prefix_size = strlen(prefix) + 1;
xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
XLogBeginInsert();
XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index d59017b..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -75,6 +76,7 @@ static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
const char *prefix, Size message_size, const char *message);
static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
Size message_size, const char *message);
/* streaming callbacks */
@@ -96,6 +98,7 @@ static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *tx
static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -1233,7 +1236,8 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- const char *prefix, Size message_size,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
@@ -1260,8 +1264,8 @@ ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
- message);
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
/* Pop the error context stack */
error_context_stack = errcallback.previous;
@@ -1585,7 +1589,8 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- const char *prefix, Size message_size,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
@@ -1616,8 +1621,8 @@ stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
- message);
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
/* Pop the error context stack */
error_context_stack = errcallback.previous;
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index ac146ba..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -881,7 +882,7 @@ void
ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
XLogRecPtr lsn, const char *prefix,
Size message_size, const char *message,
- Oid relid, NodeTag cmdtype)
+ Oid relid, DeparsedCommandType cmdtype)
{
MemoryContext oldcontext;
ReorderBufferChange *change;
@@ -2005,11 +2006,15 @@ ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
if (streaming)
rb->stream_ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
change->data.ddlmsg.message_size,
change->data.ddlmsg.message);
else
rb->ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
change->data.ddlmsg.message_size,
change->data.ddlmsg.message);
}
@@ -3777,7 +3782,7 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
sz += prefix_size + change->data.ddlmsg.message_size +
- sizeof(Size) + sizeof(Size);
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
ReorderBufferSerializeReserve(rb, sz);
data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
@@ -3785,9 +3790,13 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* might have been reallocated above */
ondisk = (ReorderBufferDiskChange *) rb->outbuf;
- /* write the prefix including the size */
+ /* write the prefix, relid and cmdtype including the size */
memcpy(data, &prefix_size, sizeof(Size));
data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
memcpy(data, change->data.ddlmsg.prefix,
prefix_size);
data += prefix_size;
@@ -4120,7 +4129,7 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
sz += prefix_size + change->data.ddlmsg.message_size +
- sizeof(Size) + sizeof(Size);
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
break;
}
@@ -4405,6 +4414,10 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.ddlmsg.prefix, data, prefix_size);
Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index a8deb33..b158580 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -56,7 +56,8 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- const char *prefix, Size sz, const char *message);
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -485,6 +486,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1705,10 +1707,14 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
/*
* Remember the xid for the message in streaming mode. See
@@ -1717,6 +1723,35 @@ pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
if (in_streaming)
xid = txn->xid;
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the relid
+ * is part of a publication that supports ddl publication. We need this because
+ * on DROP end, the relid will no longer be valid. Later on Drop end, verify that
+ * the drop is for a relid that is on the deleted_rid list, and only then send
+ * the ddl message.
+ */
+
+ if ((cmdtype == DCT_DropStart))
+ {
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+ if (relentry->pubactions.pubddl)
+ {
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+ }
+ return;
+ }
+
+ if ((cmdtype == DCT_DropEnd))
+ {
+ if(!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ {
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ }
+ }
/* Send BEGIN if we haven't yet */
if (txndata && !txndata->sent_begin_txn)
@@ -2017,7 +2052,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2075,6 +2111,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2188,6 +2225,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2373,6 +2411,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 586d6f9..7b51fb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,7 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
- pubdesc->pubactions.pubcreate |= pubform->pubcreate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7d9c6eb..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,7 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
- int i_pubcreate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3887,25 +3887,25 @@ getPublications(Archive *fout, int *numPublications)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubcreate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubcreate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3921,7 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
- i_pubcreate = PQfnumber(res, "pubcreate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3945,8 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
- pubinfo[i].pubcreate =
- (strcmp(PQgetvalue(res, i, i_pubcreate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4026,12 +4026,12 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
- if (pubinfo->pubcreate)
+ if (pubinfo->pubddl)
{
if (!first)
appendPQExpBufferStr(query, ", ");
- appendPQExpBufferStr(query, "create");
+ appendPQExpBufferStr(query, "ddl");
first = false;
}
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 170271e..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,7 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
- bool pubcreate;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 32cae61..fe28abb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6087,8 +6087,8 @@ listPublications(const char *pattern)
gettext_noop("Via root"));
if (pset.sversion >= 140000)
appendPQExpBuffer(&buf,
- ",\n pubcreate AS \"%s\"",
- gettext_noop("Creates"));
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6176,7 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
- bool has_pubcreate;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6193,7 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
- has_pubcreate = (pset.sversion >= 150000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6207,9 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
- if (has_pubcreate)
+ if (has_pubddl)
appendPQExpBufferStr(&buf,
- ", pubcreate");
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6258,7 +6258,7 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
- if (has_pubcreate)
+ if (has_pubddl)
ncols++;
initPQExpBuffer(&title);
@@ -6274,8 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
- if (has_pubcreate)
- printTableAddHeader(&cont, gettext_noop("Creates"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6286,7 +6286,7 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
- if (has_pubcreate)
+ if (has_pubddl)
printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index cf39fee..ffcd83c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11893,5 +11893,8 @@
{ oid => '4644', descr => 'trigger for ddl command deparse',
proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_ddl_deparse' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_ddl_deparse_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_start' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index bfa63cd..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -56,7 +56,7 @@ CATALOG(pg_publication,6104,PublicationRelationId)
bool pubviaroot;
/* true if table creations are published */
- bool pubcreate;
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -75,7 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
- bool pubcreate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
index 8d9be7c..172d091 100644
--- a/src/include/replication/ddlmessage.h
+++ b/src/include/replication/ddlmessage.h
@@ -15,6 +15,18 @@
#include "access/xlogreader.h"
#include "nodes/nodes.h"
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_Create,
+ DCT_DropStart,
+ DCT_DropEnd,
+ DCT_Alter
+} DeparsedCommandType;
+
/*
* Generic logical decoding DDL message wal record.
*/
@@ -33,7 +45,7 @@ typedef struct xl_logical_ddl_message
#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
-extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, NodeTag cmdtype,
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
const char *ddl_message, size_t size);
/* RMGR API*/
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index fa7a8b8..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -95,6 +95,8 @@ typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
Size message_size,
const char *message);
@@ -217,6 +219,8 @@ typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
Size message_size,
const char *message);
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 826c9ce..3664873 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -12,6 +12,7 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -447,6 +448,8 @@ typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
Size sz,
const char *message);
@@ -522,6 +525,8 @@ typedef void (*ReorderBufferStreamDDLMessageCB) (
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
Size sz,
const char *message);
@@ -668,7 +673,7 @@ extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot s
Size message_size, const char *message);
extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
const char *prefix, Size message_size,
- const char *message, Oid relid, NodeTag cmdtype);
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
index e979ee5..fb174d2 100644
--- a/src/include/tcop/ddl_deparse.h
+++ b/src/include/tcop/ddl_deparse.h
@@ -5,5 +5,6 @@
extern char *deparse_utility_command(CollectedCommand *cmd);
extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity);
#endif /* DDL_DEPARSE_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index a8a1239..b1921cc 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
-------+-------+------------+---------+---------+---------+-----------+----------+---------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 6e486cf..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,18 +30,18 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
@@ -87,9 +87,9 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
@@ -99,9 +99,9 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
@@ -109,9 +109,9 @@ Tables:
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,9 +134,9 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,9 +159,9 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | t | t | t | f | f | f | f
(1 row)
@@ -174,18 +174,18 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,9 +207,9 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,9 +223,9 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,9 +255,9 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
@@ -271,9 +271,9 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
@@ -290,9 +290,9 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
@@ -301,9 +301,9 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,9 +337,9 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
@@ -350,9 +350,9 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
@@ -676,9 +676,9 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,9 +821,9 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,9 +1029,9 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -1070,9 +1070,9 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -1151,9 +1151,9 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | f | f | f
(1 row)
@@ -1164,9 +1164,9 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
--------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
@@ -1174,9 +1174,9 @@ ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
------------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
@@ -1193,18 +1193,18 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1219,43 +1219,43 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,9 +1290,9 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1301,9 +1301,9 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
@@ -1311,9 +1311,9 @@ Tables from schemas:
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1322,9 +1322,9 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1334,9 +1334,9 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1346,9 +1346,9 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1357,9 +1357,9 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,9 +1368,9 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,9 +1379,9 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1389,18 +1389,18 @@ Tables from schemas:
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1410,9 +1410,9 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1422,9 +1422,9 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,17 +1504,17 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,9 +1525,9 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
@@ -1535,9 +1535,9 @@ Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
--
1.8.3.1
[application/octet-stream] v5-0002-Support-DDL-replication.patch (102.9K, ../../CAFPTHDaBodoZ5c7U1uyokbvq+zUvhJ4ps-7H66nHGw45UnO0OQ@mail.gmail.com/4-v5-0002-Support-DDL-replication.patch)
download | inline diff:
From e584606f51d390c67790b920b42a68f9103ac10d Mon Sep 17 00:00:00 2001
From: Ajin Cherian <ajinc@fast.au.fujitsu.com>
Date: Wed, 25 May 2022 05:18:30 -0400
Subject: [PATCH v5 2/3] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to a simple CREATE TABLE command.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/commands/event_trigger.c | 45 +++
src/backend/commands/publicationcmds.c | 50 +++
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 84 +++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 88 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 126 +++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 39 ++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 3 +
src/include/catalog/pg_publication.h | 4 +
src/include/replication/ddlmessage.h | 45 +++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 23 ++
src/include/replication/reorderbuffer.h | 34 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
29 files changed, 1164 insertions(+), 209 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..9c669c4 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -2180,3 +2183,45 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+ static int i;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Fixme: This is just for the development purpose and needs to be
+ * removed later.
+ */
+ elog(LOG, "DDL command no. %d", ++i);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, cmd->type,
+ json_string, strlen(json_string) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..fccd531 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubcreate = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "create") == 0)
+ pubactions->pubcreate = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -756,6 +761,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -820,6 +826,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubcreate - 1] =
+ BoolGetDatum(pubactions.pubcreate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +890,45 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (stmt->for_all_tables && pubactions.pubcreate)
+ {
+ CreateEventTrigStmt *ddl_trigg;
+ Node *args = NULL;
+ List *tags = NIL;
+ Oid event_trig_id;
+ char trigger_name[NAMEDATALEN];
+
+ ddl_trigg = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), "pg_deparse_trig_%u",
+ puboid);
+ ddl_trigg->trigname = pstrdup(trigger_name);
+ ddl_trigg->eventname = "ddl_command_end";
+ ddl_trigg->funcname = SystemFuncName("publication_ddl_deparse");
+
+ args = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ tags = list_make1(args);
+ ddl_trigg->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+ event_trig_id = CreateEventTrigger(ddl_trigg);
+
+ /*
+ * Register the event trigger as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, event_trig_id);
+ recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1069,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubcreate - 1] = BoolGetDatum(pubactions.pubcreate);
+ replaces[Anum_pg_publication_pubcreate - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..c40cd62
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,84 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, NodeTag cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..d59017b 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,9 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +93,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +225,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +242,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +260,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1231,43 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1583,47 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, message_size,
+ message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..ac146ba 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +875,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, NodeTag cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1996,25 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2392,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3771,36 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4115,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4382,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4398,29 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9..233abcd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 42c06af..a8deb33 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,9 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +259,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +276,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1656,8 +1661,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1702,36 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9..586d6f9 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubcreate |= pubform->pubcreate;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..7d9c6eb 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubcreate;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubcreate, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubcreate, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubcreate = PQfnumber(res, "pubcreate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubcreate =
+ (strcmp(PQgetvalue(res, i, i_pubcreate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubcreate)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "create");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..170271e 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubcreate;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924..32cae61 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubcreate AS \"%s\"",
+ gettext_noop("Creates"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubcreate;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubcreate = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubcreate)
+ appendPQExpBufferStr(&buf,
+ ", pubcreate");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubcreate)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubcreate)
+ printTableAddHeader(&cont, gettext_noop("Creates"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubcreate)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c65bcf3..cf39fee 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,5 +11890,8 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..bfa63cd 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubcreate;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubcreate;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..8d9be7c
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,45 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ NodeTag cmdtype;/* type of sql command */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, NodeTag cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..fa7a8b8 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,16 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +210,17 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +240,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +259,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..826c9ce 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,7 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +57,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +132,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ NodeTag cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +442,14 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +516,15 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +570,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +590,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +666,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, NodeTag cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2a38a93..a8a1239 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+------+-------+------------+---------+---------+---------+-----------+----------+---------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..6e486cf 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Creates
+--------------------------+------------+---------+---------+---------+-----------+----------+---------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-26 08:35 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-26 08:35 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Sat, May 14, 2022 at 6:02 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> > > 4. The handling related to partition tables seems missing because, on
> > > the subscriber-side, it always creates a relation entry in
> > > pg_subscription_rel which won't work. Check its interaction with
> > > publish_via_partition_root.
> >
> > I will test it out.
>
> Hi,
>
> patch 0010 properly handles partitioned table creation on the apply
> worker. Whether a replicated partitioned table should be added to
> pg_subscription_rel catalog depends on the setting of
> publish_via_partition_root of the publication. Thus we need to connect
> to the source DB and check if the partitioned table is in
> pg_catalog.pg_publication_tables after the apply worker creates the
> partitioned table.
>
> Thanks to Borui Yang for enabling and testing replication of DDL type
> T_RenameStmt in patch 0009.
>
> I've also rebased all the patches. Github branch of the same change
> can be found here:
> https://github.com/zli236/postgres/commits/ddl_replication
Thank you for updating the patches!
I've not looked at these patches in-depth yet but with this approach,
what do you think we can handle the DDL syntax differences between
major versions? DDL syntax or behavior could be changed by future
changes and I think we need to somehow deal with the differences. For
example, if the user uses logical replication for major version
upgrade, the publisher is older than the subscriber. We might have to
rewrite the DDL before applying to the subscriber because the DDL
executed on the publisher no longer work on a new PostgreSQL version
or we might have to add some options to the DDL before the application
in order to keep the same behavior. This seems to require a different
solution from what the patch does for the problem you mentioned such
as "DDL involving multiple tables where only some tables are
replicated”.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-26 22:19 Zheng Li <zhengli10@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-05-26 22:19 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Hi Masahiko,
> Thank you for updating the patches!
>
> I've not looked at these patches in-depth yet but with this approach,
> what do you think we can handle the DDL syntax differences between
> major versions? DDL syntax or behavior could be changed by future
> changes and I think we need to somehow deal with the differences. For
> example, if the user uses logical replication for major version
> upgrade, the publisher is older than the subscriber. We might have to
> rewrite the DDL before applying to the subscriber because the DDL
> executed on the publisher no longer work on a new PostgreSQL version
I don't think we will allow this kind of situation to happen in the
first place for
backward compatibility. If a DDL no longer works on a new version of
PostgreSQL, the user will have to change the application code as well.
So even if it happens for
whatever reason, we could either
1. fail the apply worker and let the user fix such DDL because they'll
have to fix the application code anyway when this happens.
2. add guard rail logic in the apply worker to automatically fix such
DDL if possible, knowing the version of the source and target. Similar
logic must have been implemented for pg_dump/restore/upgrade.
> or we might have to add some options to the DDL before the application
> in order to keep the same behavior. This seems to require a different
> solution from what the patch does for the problem you mentioned such
> as "DDL involving multiple tables where only some tables are
> replicated”.
First of all, this case can only happen when the customer chooses to
only replicate a subset of the tables in a database in which case
table level DDL replication is chosen instead of database level DDL
replication (where all tables
and DDLs are replicated). I think the solution would be:
1. make best effort to detect such DDLs on the publisher and avoid
logging of such DDLs in table level DDL replication.
2. apply worker will fail to replay such command due to missing
objects if such DDLs didn't get filtered on the publisher for some
reason. This should be rare and I think it's OK even if it happens,
we'll find out
why and fix it.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-27 02:03 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-05-27 02:03 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, May 27, 2022 at 3:49 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hi Masahiko,
>
> > Thank you for updating the patches!
> >
> > I've not looked at these patches in-depth yet but with this approach,
> > what do you think we can handle the DDL syntax differences between
> > major versions? DDL syntax or behavior could be changed by future
> > changes and I think we need to somehow deal with the differences. For
>
> > example, if the user uses logical replication for major version
> > upgrade, the publisher is older than the subscriber. We might have to
> > rewrite the DDL before applying to the subscriber because the DDL
> > executed on the publisher no longer work on a new PostgreSQL version
>
> I don't think we will allow this kind of situation to happen in the
> first place for
> backward compatibility. If a DDL no longer works on a new version of
> PostgreSQL, the user will have to change the application code as well.
> So even if it happens for
> whatever reason, we could either
> 1. fail the apply worker and let the user fix such DDL because they'll
> have to fix the application code anyway when this happens.
> 2. add guard rail logic in the apply worker to automatically fix such
> DDL if possible, knowing the version of the source and target. Similar
> logic must have been implemented for pg_dump/restore/upgrade.
>
> > or we might have to add some options to the DDL before the application
> > in order to keep the same behavior. This seems to require a different
> > solution from what the patch does for the problem you mentioned such
>
> > as "DDL involving multiple tables where only some tables are
> > replicated”.
>
> First of all, this case can only happen when the customer chooses to
> only replicate a subset of the tables in a database in which case
> table level DDL replication is chosen instead of database level DDL
> replication (where all tables
> and DDLs are replicated). I think the solution would be:
> 1. make best effort to detect such DDLs on the publisher and avoid
> logging of such DDLs in table level DDL replication.
> 2. apply worker will fail to replay such command due to missing
> objects if such DDLs didn't get filtered on the publisher for some
> reason. This should be rare and I think it's OK even if it happens,
> we'll find out
> why and fix it.
>
FWIW, both these cases could be handled with the deparsing approach,
and the handling related to the drop of multiple tables where only a
few are published is already done in the last POC patch shared by Ajin
[1].
[1] - https://www.postgresql.org/message-id/CAFPTHDaBodoZ5c7U1uyokbvq%2BzUvhJ4ps-7H66nHGw45UnO0OQ%40mail.g...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-05-27 09:07 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Ajin Cherian <itsajin@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-05-27 09:07 UTC (permalink / raw)
To: Ajin Cherian <itsajin@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; Zheng Li <zhengli10@gmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; rajesh.rs0541@gmail.com <rajesh.rs0541@gmail.com>
On Thursday, May 26, 2022 1:46 PM Ajin Cherian <itsajin@gmail.com> wrote:
>
> On Tue, May 10, 2022 at 9:33 PM Ajin Cherian <itsajin@gmail.com> wrote:
> >
> > On Fri, May 6, 2022 at 11:24 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> > > As we have hacked CreatePublication function for this POC, the
> > > regression tests are not passing but we can easily change it so that
> > > we invoke new functionality with the syntax proposed in this thread
> > > or with some other syntax and we shall do that in the next patch
> > > unless this approach is not worth pursuing.
> > >
> > > This POC is prepared by Ajin Cherian, Hou-San, and me.
> > >
> > > Thoughts?
> > >
> > > [1] -
> > >
> https://www.postgresql.org/message-id/20150215044814.GL3391%40alvh.n
> > > o-ip.org
> >
> > I have updated Amit's patch by including a public action "create" when
> > creating publication which is turned off by default.
> > Now the 'make check' tests pass. I also fixed a problem that failed to
> > create tables when the table has a primary key.
>
> I have updated this "deparse" patch-set to include support for 'drop table':
>
Here is the new version POC patches which add support for 'ALTER TABLE'
For non-rewrite ALTER TABLE command:
we deparse the command and WAL log the deparsed json string with the relid
of the altered table at table_rewrite event trigger. The WALSender decodes
the WAL and sends it to subscriber if the altered table is published. Most
of ALTER TABLE command are supported except some commands(DDL related to
PARTITIONED TABLE ...) that introduced recently which haven't been
supported by the current ddl_deparser, we will support that later.
For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string with the relid
of the altered table at table_rewrite event trigger. The WALSender decodes
the WAL and sends it to subscriber if the altered table is published.
Then, the WALSender will convert the upcoming rewrite INSERTs to UPDATEs
and send them to subscriber so that the data between publisher and
subscriber can always be consistent. Note that the tables that publish
rewrite ddl must have a replica identity configured in order to be able to
replicate the upcoming rewrite UPDATEs.
We do this way because of two reasons:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
Here is the example for the rewrite ddl: ALTER TABLE ADD COLUMN DEFAULT:
A table on the publisher side has rows:
ddl_test(a)
a
----
1
2
The same table on the subscriber side has rows:
ddl_test(a)
a
----
1
2
3
4
-----------------------------------
If we execute "ALTER TABLE ddl_test ADD COLUMN b int DEFAULT random();" on
publisher. The row(1,2) on subscriber will be updated by the rewrite
UPDATE received from publisher. The row(3,4) will be updated by exectuing
the "ADD COLUMN b int DEFAULT random();" on the subscriber.
-----------------------------------
ddl_test(a,b)
a|b
----
1|random num(pub)
2|random num(pub)
The same table on the subscriber side has rows:
ddl_test(a,b)
a|b
----
1|random num(pub)
2|random num(pub)
3|random num(sub)
4|random num(sub)
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v5-0001-Functions-to-deparse-DDL-commands.patch (85.8K, ../../OS0PR01MB5716D74348FB1F76008B31CB94D89@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v5-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 275544b1a7f7e5d68f22c66ffd21b0c3e2dccc07 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE/ALTER TABLE/DROP TABLE specific
functions. Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 2180 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 +++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 7 +-
src/include/tcop/ddl_deparse.h | 10 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2957 insertions(+), 1 deletion(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..60c077f
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,2180 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..fb174d2
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,10 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
2.7.2.windows.1
[application/octet-stream] v5-0002-Support-DDL-replication.patch (123.7K, ../../OS0PR01MB5716D74348FB1F76008B31CB94D89@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v5-0002-Support-DDL-replication.patch)
download | inline diff:
From c3e7bde630a1e37846f7ce56e36cbb8274ae550c Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 14:00:07 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER TABLE command and CREATE TABLE:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
- For DROP TABLE:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 146 ++++++++-
src/backend/commands/publicationcmds.c | 104 +++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 164 +++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 57 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/include/tcop/deparse_utility.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
34 files changed, 1491 insertions(+), 217 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..9bc2145 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,142 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_DropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_ddl_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_ddl_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_Alter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_Alter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_Create;
+ }
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ SQLDropObject *obj;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ char *command;
+
+ command = deparse_drop_table(obj->objidentity);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_DropEnd,
+ command, strlen(command) + 1);
+ }
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..63ffc62 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -756,6 +761,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced_start, referenced_end,
+ referenced_table_rewrite;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -820,6 +827,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +891,98 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CreateEventTrigStmt *ddl_trigg_start, *ddl_trigg_end,
+ *ddl_trigg_table_rewrite;
+ Node *end_arg1 = NULL;
+ Node *start_arg1 = NULL;
+ Node *table_rewrite_arg1 = NULL;
+ Node *arg2 = NULL;
+ Node *arg3 = NULL;
+ List *end_tags = NIL;
+ List *start_tags = NIL;
+ List *table_rewrite_tags = NIL;
+ Oid event_trig_start_id, event_trig_end_id,
+ event_trig_table_rewrite_id;
+ char trigger_name_start[NAMEDATALEN];
+ char trigger_name_end[NAMEDATALEN];
+ char trigger_name_table_rewrite[NAMEDATALEN];
+
+ ddl_trigg_end = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_end, sizeof(trigger_name_end), "pg_deparse_trig_end_%u",
+ puboid);
+ ddl_trigg_end->trigname = pstrdup(trigger_name_end);
+ ddl_trigg_end->eventname = "ddl_command_end";
+ ddl_trigg_end->funcname = SystemFuncName("publication_ddl_deparse");
+
+ end_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ end_tags = list_make1(end_arg1);
+
+ /* support create table only when publication is for all tables */
+ if (stmt->for_all_tables)
+ {
+ arg2 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ end_tags = lappend(end_tags, arg2);
+ }
+
+ arg3 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_ALTER_TABLE)));
+ end_tags = lappend(end_tags, arg3);
+
+ ddl_trigg_end->whenclause = list_make1(makeDefElem("tag", (Node *) end_tags, -1));
+ event_trig_end_id = CreateEventTrigger(ddl_trigg_end);
+
+ ddl_trigg_start = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_start, sizeof(trigger_name_start), "pg_deparse_trig_start_%u",
+ puboid);
+ ddl_trigg_start->trigname = pstrdup(trigger_name_start);
+ ddl_trigg_start->eventname = "ddl_command_start";
+ ddl_trigg_start->funcname = SystemFuncName("publication_ddl_deparse_start");
+
+ start_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ start_tags = list_make1(start_arg1);
+ ddl_trigg_start->whenclause = list_make1(makeDefElem("tag", (Node *) start_tags, -1));
+ event_trig_start_id = CreateEventTrigger(ddl_trigg_start);
+
+ ddl_trigg_table_rewrite = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_table_rewrite, sizeof(trigger_name_table_rewrite),
+ "pg_deparse_trig_table_rewrite_%u", puboid);
+ ddl_trigg_table_rewrite->trigname = pstrdup(trigger_name_table_rewrite);
+ ddl_trigg_table_rewrite->eventname = "table_rewrite";
+ ddl_trigg_table_rewrite->funcname = SystemFuncName("publication_ddl_deparse_table_rewrite");
+
+ table_rewrite_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_ALTER_TABLE)));
+ table_rewrite_tags = list_make1(table_rewrite_arg1);
+ ddl_trigg_table_rewrite->whenclause = list_make1(makeDefElem("tag", (Node *) table_rewrite_tags, -1));
+ event_trig_table_rewrite_id = CreateEventTrigger(ddl_trigg_table_rewrite);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced_end, EventTriggerRelationId, event_trig_end_id);
+ recordDependencyOn(&referenced_end, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_start, EventTriggerRelationId, event_trig_start_id);
+ recordDependencyOn(&referenced_start, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_table_rewrite, EventTriggerRelationId, event_trig_table_rewrite_id);
+ recordDependencyOn(&referenced_table_rewrite, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1123,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..74108b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5218,7 +5218,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9..233abcd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 42c06af..8b2648b 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +260,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +277,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +413,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +487,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1370,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1419,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1455,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1476,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1618,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1698,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1739,110 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the relid
+ * is part of a publication that supports ddl publication. We need this because
+ * on DROP end, the relid will no longer be valid. Later on Drop end, verify that
+ * the drop is for a relid that is on the deleted_rid list, and only then send
+ * the ddl message.
+ */
+
+ if ((cmdtype == DCT_DropStart))
+ {
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+ if (relentry->pubactions.pubddl)
+ {
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+ }
+ return;
+ }
+ else if ((cmdtype == DCT_DropEnd))
+ {
+ if(!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ {
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ }
+ }
+ else if (cmdtype == DCT_Alter)
+ {
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ return;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2128,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2187,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2301,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2487,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9..7b51fb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924..fe28abb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..6396566 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_ddl_deparse_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_ddl_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..172d091
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,57 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_Create,
+ DCT_DropStart,
+ DCT_DropEnd,
+ DCT_Alter
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ NodeTag cmdtype;/* type of sql command */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..3664873 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ NodeTag cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2a38a93..b1921cc 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-27 16:15 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 0 replies; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-27 16:15 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, May 27, 2022 at 7:19 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hi Masahiko,
>
> > Thank you for updating the patches!
> >
> > I've not looked at these patches in-depth yet but with this approach,
> > what do you think we can handle the DDL syntax differences between
> > major versions? DDL syntax or behavior could be changed by future
> > changes and I think we need to somehow deal with the differences. For
>
> > example, if the user uses logical replication for major version
> > upgrade, the publisher is older than the subscriber. We might have to
> > rewrite the DDL before applying to the subscriber because the DDL
> > executed on the publisher no longer work on a new PostgreSQL version
>
> I don't think we will allow this kind of situation to happen in the
> first place for
> backward compatibility.
It seems like a big limitation to me.
> If a DDL no longer works on a new version of
> PostgreSQL, the user will have to change the application code as well.
> So even if it happens for
> whatever reason, we could either
> 1. fail the apply worker and let the user fix such DDL because they'll
> have to fix the application code anyway when this happens.
Once the apply worker received the DDL, if the DDL doesn't work on the
subscriber, it will enter an infinite loop until the problem is fixed.
If the failure is due to a syntax error, how does the user fix it?
> 2. add guard rail logic in the apply worker to automatically fix such
> DDL if possible, knowing the version of the source and target. Similar
> logic must have been implemented for pg_dump/restore/upgrade.
If I'm not missing something, there is no such implementation in
pg_dump/restore/upgrade. When we use pg_dump/pg_restore for major
version upgrades, we usually use the newer version pg_dump to fetch
objects from the older version server, then restore the objects by
using the newer version pg_restore.
>
> > or we might have to add some options to the DDL before the application
> > in order to keep the same behavior. This seems to require a different
> > solution from what the patch does for the problem you mentioned such
>
> > as "DDL involving multiple tables where only some tables are
> > replicated”.
>
> First of all, this case can only happen when the customer chooses to
> only replicate a subset of the tables in a database in which case
> table level DDL replication is chosen instead of database level DDL
> replication (where all tables
> and DDLs are replicated). I think the solution would be:
> 1. make best effort to detect such DDLs on the publisher and avoid
> logging of such DDLs in table level DDL replication.
I think it's better to support this case.
> 2. apply worker will fail to replay such command due to missing
> objects if such DDLs didn't get filtered on the publisher for some
> reason. This should be rare and I think it's OK even if it happens,
> we'll find out
> why and fix it.
I'm not sure it's rare since replicating a subset of tables is a
common use case of logical replication. But even if we want to go this
way I think we should consider how to fix it at this stage, otherwise
we will end up redesigning it.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-05-30 06:52 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-05-30 06:52 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, May 27, 2022 at 11:03 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, May 27, 2022 at 3:49 AM Zheng Li <zhengli10@gmail.com> wrote:
> >
> > Hi Masahiko,
> >
> > > Thank you for updating the patches!
> > >
> > > I've not looked at these patches in-depth yet but with this approach,
> > > what do you think we can handle the DDL syntax differences between
> > > major versions? DDL syntax or behavior could be changed by future
> > > changes and I think we need to somehow deal with the differences. For
> >
> > > example, if the user uses logical replication for major version
> > > upgrade, the publisher is older than the subscriber. We might have to
> > > rewrite the DDL before applying to the subscriber because the DDL
> > > executed on the publisher no longer work on a new PostgreSQL version
> >
> > I don't think we will allow this kind of situation to happen in the
> > first place for
> > backward compatibility. If a DDL no longer works on a new version of
> > PostgreSQL, the user will have to change the application code as well.
> > So even if it happens for
> > whatever reason, we could either
> > 1. fail the apply worker and let the user fix such DDL because they'll
> > have to fix the application code anyway when this happens.
> > 2. add guard rail logic in the apply worker to automatically fix such
> > DDL if possible, knowing the version of the source and target. Similar
> > logic must have been implemented for pg_dump/restore/upgrade.
> >
> > > or we might have to add some options to the DDL before the application
> > > in order to keep the same behavior. This seems to require a different
> > > solution from what the patch does for the problem you mentioned such
> >
> > > as "DDL involving multiple tables where only some tables are
> > > replicated”.
> >
> > First of all, this case can only happen when the customer chooses to
> > only replicate a subset of the tables in a database in which case
> > table level DDL replication is chosen instead of database level DDL
> > replication (where all tables
> > and DDLs are replicated). I think the solution would be:
> > 1. make best effort to detect such DDLs on the publisher and avoid
> > logging of such DDLs in table level DDL replication.
> > 2. apply worker will fail to replay such command due to missing
> > objects if such DDLs didn't get filtered on the publisher for some
> > reason. This should be rare and I think it's OK even if it happens,
> > we'll find out
> > why and fix it.
> >
>
> FWIW, both these cases could be handled with the deparsing approach,
> and the handling related to the drop of multiple tables where only a
> few are published is already done in the last POC patch shared by Ajin
> [1].
>
Right. So I'm inclined to think that deparsing approach is better from
this point as well as the point mentioned by Álvaro before[1].
Regards,
[1] https://www.postgresql.org/message-id/202204081134.6tcmf5cxl3sz%40alvherre.pgsql
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-02 12:14 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-02 12:14 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Monday, May 30, 2022 2:52 PM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Fri, May 27, 2022 at 11:03 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Fri, May 27, 2022 at 3:49 AM Zheng Li <zhengli10@gmail.com> wrote:
> > >
> > > Hi Masahiko,
> > >
> > > > Thank you for updating the patches!
> > > >
> > > > I've not looked at these patches in-depth yet but with this approach,
> > > > what do you think we can handle the DDL syntax differences between
> > > > major versions? DDL syntax or behavior could be changed by future
> > > > changes and I think we need to somehow deal with the differences. For
> > >
> > > > example, if the user uses logical replication for major version
> > > > upgrade, the publisher is older than the subscriber. We might have to
> > > > rewrite the DDL before applying to the subscriber because the DDL
> > > > executed on the publisher no longer work on a new PostgreSQL version
> > >
> > > I don't think we will allow this kind of situation to happen in the
> > > first place for
> > > backward compatibility. If a DDL no longer works on a new version of
> > > PostgreSQL, the user will have to change the application code as well.
> > > So even if it happens for
> > > whatever reason, we could either
> > > 1. fail the apply worker and let the user fix such DDL because they'll
> > > have to fix the application code anyway when this happens.
> > > 2. add guard rail logic in the apply worker to automatically fix such
> > > DDL if possible, knowing the version of the source and target. Similar
> > > logic must have been implemented for pg_dump/restore/upgrade.
> > >
> > > > or we might have to add some options to the DDL before the application
> > > > in order to keep the same behavior. This seems to require a different
> > > > solution from what the patch does for the problem you mentioned such
> > >
> > > > as "DDL involving multiple tables where only some tables are
> > > > replicated”.
> > >
> > > First of all, this case can only happen when the customer chooses to
> > > only replicate a subset of the tables in a database in which case
> > > table level DDL replication is chosen instead of database level DDL
> > > replication (where all tables
> > > and DDLs are replicated). I think the solution would be:
> > > 1. make best effort to detect such DDLs on the publisher and avoid
> > > logging of such DDLs in table level DDL replication.
> > > 2. apply worker will fail to replay such command due to missing
> > > objects if such DDLs didn't get filtered on the publisher for some
> > > reason. This should be rare and I think it's OK even if it happens,
> > > we'll find out
> > > why and fix it.
> > >
> >
> > FWIW, both these cases could be handled with the deparsing approach,
> > and the handling related to the drop of multiple tables where only a
> > few are published is already done in the last POC patch shared by Ajin
> > [1].
> >
>
> Right. So I'm inclined to think that deparsing approach is better from
> this point as well as the point mentioned by Álvaro before[1].
I agree. One more point about deparsing approach is that it can also
help to replicate CREATE TABLE AS/SELECT INTO in a better way.
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
One problem of this approach is that we cannot use the current trigger to
deparse or WAL log the CREATE TABLE. Because none of the even trigger is fired
after creating the table and before inserting the data. To solve this, one idea
is that we could directly add some code at the end of create_ctas_internal() to
deparse and WAL log it. Moreover, we could even introduce a new type of event
trigger(table_create) which would be fired at the expected timing so that we
can use the trigger function to deparse and WAL log. I am not sure which way is
better. I temporarily use the second idea which introduce a new type event
trigger in the 0003 POC patch.
In the POC patch, we deparse the command in the table_create event trigger and
WAL log the deparsed json string. The walsender will send the string to
subscriber. And incoming INSERTs will also be replicated.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v6-0001-Functions-to-deparse-DDL-commands.patch (85.8K, ../../OS0PR01MB571695EDF9EAB2422FBF2C1094DE9@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v6-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 275544b1a7f7e5d68f22c66ffd21b0c3e2dccc07 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE/ALTER TABLE/DROP TABLE specific
functions. Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 2180 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 +++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 7 +-
src/include/tcop/ddl_deparse.h | 10 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2957 insertions(+), 1 deletion(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..60c077f
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,2180 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..fb174d2
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,10 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
2.7.2.windows.1
[application/octet-stream] v6-0002-Support-DDL-replication.patch (127.5K, ../../OS0PR01MB571695EDF9EAB2422FBF2C1094DE9@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v6-0002-Support-DDL-replication.patch)
download | inline diff:
From 5a10fd3e92c86b6cd926a6bfafc10b31a1c560e1 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 14:00:07 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER TABLE command and CREATE TABLE:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
- For DROP TABLE:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 146 ++++++++-
src/backend/commands/publicationcmds.c | 104 +++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 164 +++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 57 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/include/tcop/deparse_utility.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
src/test/subscription/t/032_ddl_replication.pl | 86 ++++++
35 files changed, 1577 insertions(+), 217 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..9bc2145 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,142 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_DropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_ddl_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_ddl_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_Alter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_ddl_deparse
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_ddl_deparse(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_Alter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_Create;
+ }
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ SQLDropObject *obj;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ char *command;
+
+ command = deparse_drop_table(obj->objidentity);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_DropEnd,
+ command, strlen(command) + 1);
+ }
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..63ffc62 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -756,6 +761,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
{
Relation rel;
ObjectAddress myself;
+ ObjectAddress referenced_start, referenced_end,
+ referenced_table_rewrite;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -820,6 +827,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +891,98 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE TABLE works.
+ * We need to enhance this once the approach for DDL replication is
+ * finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CreateEventTrigStmt *ddl_trigg_start, *ddl_trigg_end,
+ *ddl_trigg_table_rewrite;
+ Node *end_arg1 = NULL;
+ Node *start_arg1 = NULL;
+ Node *table_rewrite_arg1 = NULL;
+ Node *arg2 = NULL;
+ Node *arg3 = NULL;
+ List *end_tags = NIL;
+ List *start_tags = NIL;
+ List *table_rewrite_tags = NIL;
+ Oid event_trig_start_id, event_trig_end_id,
+ event_trig_table_rewrite_id;
+ char trigger_name_start[NAMEDATALEN];
+ char trigger_name_end[NAMEDATALEN];
+ char trigger_name_table_rewrite[NAMEDATALEN];
+
+ ddl_trigg_end = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_end, sizeof(trigger_name_end), "pg_deparse_trig_end_%u",
+ puboid);
+ ddl_trigg_end->trigname = pstrdup(trigger_name_end);
+ ddl_trigg_end->eventname = "ddl_command_end";
+ ddl_trigg_end->funcname = SystemFuncName("publication_ddl_deparse");
+
+ end_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ end_tags = list_make1(end_arg1);
+
+ /* support create table only when publication is for all tables */
+ if (stmt->for_all_tables)
+ {
+ arg2 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE)));
+ end_tags = lappend(end_tags, arg2);
+ }
+
+ arg3 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_ALTER_TABLE)));
+ end_tags = lappend(end_tags, arg3);
+
+ ddl_trigg_end->whenclause = list_make1(makeDefElem("tag", (Node *) end_tags, -1));
+ event_trig_end_id = CreateEventTrigger(ddl_trigg_end);
+
+ ddl_trigg_start = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_start, sizeof(trigger_name_start), "pg_deparse_trig_start_%u",
+ puboid);
+ ddl_trigg_start->trigname = pstrdup(trigger_name_start);
+ ddl_trigg_start->eventname = "ddl_command_start";
+ ddl_trigg_start->funcname = SystemFuncName("publication_ddl_deparse_start");
+
+ start_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_DROP_TABLE)));
+ start_tags = list_make1(start_arg1);
+ ddl_trigg_start->whenclause = list_make1(makeDefElem("tag", (Node *) start_tags, -1));
+ event_trig_start_id = CreateEventTrigger(ddl_trigg_start);
+
+ ddl_trigg_table_rewrite = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_table_rewrite, sizeof(trigger_name_table_rewrite),
+ "pg_deparse_trig_table_rewrite_%u", puboid);
+ ddl_trigg_table_rewrite->trigname = pstrdup(trigger_name_table_rewrite);
+ ddl_trigg_table_rewrite->eventname = "table_rewrite";
+ ddl_trigg_table_rewrite->funcname = SystemFuncName("publication_ddl_deparse_table_rewrite");
+
+ table_rewrite_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_ALTER_TABLE)));
+ table_rewrite_tags = list_make1(table_rewrite_arg1);
+ ddl_trigg_table_rewrite->whenclause = list_make1(makeDefElem("tag", (Node *) table_rewrite_tags, -1));
+ event_trig_table_rewrite_id = CreateEventTrigger(ddl_trigg_table_rewrite);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced_end, EventTriggerRelationId, event_trig_end_id);
+ recordDependencyOn(&referenced_end, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_start, EventTriggerRelationId, event_trig_start_id);
+ recordDependencyOn(&referenced_start, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_table_rewrite, EventTriggerRelationId, event_trig_table_rewrite_id);
+ recordDependencyOn(&referenced_table_rewrite, &myself, DEPENDENCY_INTERNAL);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1123,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..74108b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5218,7 +5218,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9..233abcd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 42c06af..8b2648b 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +260,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +277,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +413,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +487,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1370,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1419,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1455,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1476,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1618,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1698,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1739,110 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the relid
+ * is part of a publication that supports ddl publication. We need this because
+ * on DROP end, the relid will no longer be valid. Later on Drop end, verify that
+ * the drop is for a relid that is on the deleted_rid list, and only then send
+ * the ddl message.
+ */
+
+ if ((cmdtype == DCT_DropStart))
+ {
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+ if (relentry->pubactions.pubddl)
+ {
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+ }
+ return;
+ }
+ else if ((cmdtype == DCT_DropEnd))
+ {
+ if(!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ {
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ }
+ }
+ else if (cmdtype == DCT_Alter)
+ {
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ return;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2128,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2187,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2301,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2487,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9..7b51fb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924..fe28abb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..6396566 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_ddl_deparse', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_ddl_deparse_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_ddl_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..172d091
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,57 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_Create,
+ DCT_DropStart,
+ DCT_DropEnd,
+ DCT_Alter
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ NodeTag cmdtype;/* type of sql command */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..3664873 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ NodeTag cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2a38a93..b1921cc 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..c36a7c0
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,86 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test ALTER TABLE command is replicated on table test_rep
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$node_publisher->wait_for_catchup('mysub');
+
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE t4;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(0), 'TABLE t4 is dropped');
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.7.2.windows.1
[application/octet-stream] v6-0003-support-CREATE-TABLE-AS-SELECT-INTO.patch (14.9K, ../../OS0PR01MB571695EDF9EAB2422FBF2C1094DE9@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v6-0003-support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From cdd09f1ac681b3d0d5bf88c0bfa920896dd4b3be Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 2 Jun 2022 13:32:39 +0800
Subject: [PATCH] support CREATE TABLE AS/SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_create". which would be fired
for CREATE TABLE/CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_create event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 3 +
src/backend/commands/event_trigger.c | 161 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 33 ++++++-
src/backend/tcop/utility.c | 7 ++
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/utils/evtcache.h | 3 +-
8 files changed, 208 insertions(+), 8 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..ae25d2a 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ EventTriggerTableCreate((Node *) create, intoRelationAddr);
return intoRelationAddr;
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 9bc2145..e77f01b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_create") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_create") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -586,7 +588,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableCreate)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -869,6 +872,155 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerCreateTableStart
+ * Prepare to receive data on an CREATE TABLE [AS] command about to be
+ * executed.
+ */
+void
+EventTriggerCreateTableStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_Simple;
+ command->in_extension = creating_extension;
+ command->d.simple.address = InvalidObjectAddress;
+ command->d.simple.secondaryObject = InvalidObjectAddress;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerCreateTableEnd
+ * Finish up saving an CREATE TABLE [AS] command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerCreateTableEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_ddl_deparse_table_create
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_ddl_deparse_table_create(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_Create,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_rewrite triggers.
+ */
+void
+EventTriggerTableCreate(Node *parsetree, ObjectAddress address)
+{
+ List *runlist;
+ Node *old_parsetree;
+ EventTriggerData trigdata;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ runlist = EventTriggerCommonSetup(currentEventTriggerState->currentCommand->parsetree,
+ EVT_TableCreate,
+ "table_create",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /*
+ * Use PG_TRY to ensure parsetree is reset even when one trigger
+ * fails. (This is perhaps not necessary, as the currentState variable will
+ * be removed shortly by our caller, but it seems better to play safe.)
+ */
+ old_parsetree = currentEventTriggerState->currentCommand->parsetree;
+ currentEventTriggerState->currentCommand->d.simple.address = address;
+ currentEventTriggerState->currentCommand->parsetree = parsetree;
+
+ /* Run the triggers. */
+ PG_TRY();
+ {
+ EventTriggerInvoke(runlist, &trigdata);
+ }
+ PG_FINALLY();
+ {
+ currentEventTriggerState->currentCommand->parsetree = old_parsetree;
+ }
+ PG_END_TRY();
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1149,7 +1301,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableCreate)) > 0;
}
/*
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 63ffc62..187e326 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -762,7 +762,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Relation rel;
ObjectAddress myself;
ObjectAddress referenced_start, referenced_end,
- referenced_table_rewrite;
+ referenced_table_rewrite,
+ referenced_table_create;
Oid puboid;
bool nulls[Natts_pg_publication];
Datum values[Natts_pg_publication];
@@ -904,20 +905,25 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
if (pubactions.pubddl)
{
CreateEventTrigStmt *ddl_trigg_start, *ddl_trigg_end,
- *ddl_trigg_table_rewrite;
+ *ddl_trigg_table_rewrite,
+ *ddl_trigg_table_create;
Node *end_arg1 = NULL;
Node *start_arg1 = NULL;
Node *table_rewrite_arg1 = NULL;
+ Node *table_create_arg1 = NULL;
Node *arg2 = NULL;
Node *arg3 = NULL;
List *end_tags = NIL;
List *start_tags = NIL;
List *table_rewrite_tags = NIL;
+ List *table_create_tags = NIL;
Oid event_trig_start_id, event_trig_end_id,
- event_trig_table_rewrite_id;
+ event_trig_table_rewrite_id,
+ event_trig_table_create_id;
char trigger_name_start[NAMEDATALEN];
char trigger_name_end[NAMEDATALEN];
char trigger_name_table_rewrite[NAMEDATALEN];
+ char trigger_name_table_create[NAMEDATALEN];
ddl_trigg_end = makeNode(CreateEventTrigStmt);
@@ -969,6 +975,24 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
ddl_trigg_table_rewrite->whenclause = list_make1(makeDefElem("tag", (Node *) table_rewrite_tags, -1));
event_trig_table_rewrite_id = CreateEventTrigger(ddl_trigg_table_rewrite);
+ /* Trigger for table_create */
+ ddl_trigg_table_create = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name_table_create, sizeof(trigger_name_table_create),
+ "pg_deparse_trig_table_create_%u", puboid);
+ ddl_trigg_table_create->trigname = pstrdup(trigger_name_table_create);
+ ddl_trigg_table_create->eventname = "table_create";
+ ddl_trigg_table_create->funcname = SystemFuncName("publication_ddl_deparse_table_create");
+
+ table_create_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_CREATE_TABLE_AS)));
+ table_create_tags = list_make1(table_create_arg1);
+ table_create_arg1 = (Node *) makeString(pstrdup(GetCommandTagName(CMDTAG_SELECT_INTO)));
+ table_create_tags = lappend(table_create_tags, table_create_arg1);
+
+ ddl_trigg_table_create->whenclause = list_make1(makeDefElem("tag", (Node *) table_create_tags, -1));
+ event_trig_table_create_id = CreateEventTrigger(ddl_trigg_table_create);
+
+
/*
* Register the event triggers as internally dependent on the
* publication.
@@ -981,6 +1005,9 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
ObjectAddressSet(referenced_table_rewrite, EventTriggerRelationId, event_trig_table_rewrite_id);
recordDependencyOn(&referenced_table_rewrite, &myself, DEPENDENCY_INTERNAL);
+
+ ObjectAddressSet(referenced_table_create, EventTriggerRelationId, event_trig_table_create_id);
+ recordDependencyOn(&referenced_table_create, &myself, DEPENDENCY_INTERNAL);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..c16f804 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1164,6 +1164,8 @@ ProcessUtilitySlow(ParseState *pstate,
Datum toast_options;
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
+ EventTriggerCreateTableStart(parsetree);
+
/* Remember transformed RangeVar for LIKE */
table_rv = cstmt->relation;
@@ -1198,6 +1200,9 @@ ProcessUtilitySlow(ParseState *pstate,
NewRelationCreateToastTable(address.objectId,
toast_options);
+
+ EventTriggerTableCreate((Node *) cstmt, address);
+ EventTriggerCreateTableEnd();
}
else if (IsA(stmt, CreateForeignTableStmt))
{
@@ -1665,8 +1670,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerCreateTableStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerCreateTableEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index 3a9c9f0..f4c8b73 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_create") == 0)
+ event = EVT_TableCreate;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6396566..f55f455 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11899,4 +11899,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_ddl_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_ddl_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_ddl_deparse_table_create', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_ddl_deparse_table_create' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..d617e94 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerCreateTableStart(Node *parsetree);
+extern void EventTriggerTableCreate(Node *parsetree, ObjectAddress address);
+extern void EventTriggerCreateTableEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..070c6b9 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableCreate
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-03 11:15 Amit Kapila <amit.kapila16@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-03 11:15 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Thu, Jun 2, 2022 at 5:44 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
> TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
> after creating the table and before writing data into the table and replicate
> the incoming writes later as normal INSERTs. In this apporach, we don't execute
> the subquery on subscriber so that don't need to make sure all the objects
> referenced in the subquery also exists in subscriber. And This approach works
> for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
>
> One problem of this approach is that we cannot use the current trigger to
> deparse or WAL log the CREATE TABLE. Because none of the even trigger is fired
> after creating the table and before inserting the data. To solve this, one idea
> is that we could directly add some code at the end of create_ctas_internal() to
> deparse and WAL log it. Moreover, we could even introduce a new type of event
> trigger(table_create) which would be fired at the expected timing so that we
> can use the trigger function to deparse and WAL log. I am not sure which way is
> better.
>
I am also not able to think of a better way to replicate CREATE TABLE
AS/SELECT INTO other than to use a new type of event trigger. I think
it is better to discuss this new type of event trigger in a separate
thread with the use case of DDL replication unless we have a better
idea to achieve this.
Few comments:
===============
1. Why not capture the CREATE TABLE/CREATE TABLE AS ... command with
EventTriggerCollectSimpleCommand instead of using
EventTriggerCreateTableStart?
2.
+ /*
+ * Use PG_TRY to ensure parsetree is reset even when one trigger
+ * fails. (This is perhaps not necessary, as the currentState variable will
+ * be removed shortly by our caller, but it seems better to play safe.)
+ */
+ old_parsetree = currentEventTriggerState->currentCommand->parsetree;
+ currentEventTriggerState->currentCommand->d.simple.address = address;
+ currentEventTriggerState->currentCommand->parsetree = parsetree;
Instead of doing this can't we use the parsetree stored in trigdata to
deparse the statement?
3. Is there a reason to invoke the trigger after defining the relation
instead of doing it before similar to table_rewrite trigger
(EventTriggerTableRewrite).
4. It should be published for all tables publication similar to 'create table'
5. The new code in CreatePublication looks quite haphazard, can we
improve it by moving it into separate functions?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-07 02:59 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-07 02:59 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Friday, June 3, 2022 7:16 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Thu, Jun 2, 2022 at 5:44 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > The main idea of replicating the CREATE TABLE AS is that we deprase
> > the CREATE TABLE AS into a simple CREATE TABLE(without subquery)
> > command and WAL log it after creating the table and before writing
> > data into the table and replicate the incoming writes later as normal
> > INSERTs. In this apporach, we don't execute the subquery on subscriber
> > so that don't need to make sure all the objects referenced in the
> > subquery also exists in subscriber. And This approach works for all
> > kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
> >
> > One problem of this approach is that we cannot use the current trigger
> > to deparse or WAL log the CREATE TABLE. Because none of the even
> > trigger is fired after creating the table and before inserting the
> > data. To solve this, one idea is that we could directly add some code
> > at the end of create_ctas_internal() to deparse and WAL log it.
> > Moreover, we could even introduce a new type of event
> > trigger(table_create) which would be fired at the expected timing so
> > that we can use the trigger function to deparse and WAL log. I am not
> > sure which way is better.
> >
>
> I am also not able to think of a better way to replicate CREATE TABLE AS/SELECT
> INTO other than to use a new type of event trigger. I think it is better to discuss
> this new type of event trigger in a separate thread with the use case of DDL
> replication unless we have a better idea to achieve this.
>
> Few comments:
> ===============
> 1. Why not capture the CREATE TABLE/CREATE TABLE AS ... command with
> EventTriggerCollectSimpleCommand instead of using
> EventTriggerCreateTableStart?
After thinking more, I remove this part as it seems enough for the new type
trigger to only catch the CTAS and SELECT INTO.
> 2.
> + /*
> + * Use PG_TRY to ensure parsetree is reset even when one trigger
> + * fails. (This is perhaps not necessary, as the currentState variable
> + will
> + * be removed shortly by our caller, but it seems better to play safe.)
> + */ old_parsetree =
> + currentEventTriggerState->currentCommand->parsetree;
> + currentEventTriggerState->currentCommand->d.simple.address = address;
> + currentEventTriggerState->currentCommand->parsetree = parsetree;
>
> Instead of doing this can't we use the parsetree stored in trigdata to deparse
> the statement?
We need to use the real createstmt which contains the actual column info. But I agree
It looks hacky and I adjusted this part in the patch.
> 3. Is there a reason to invoke the trigger after defining the relation instead of
> doing it before similar to table_rewrite trigger (EventTriggerTableRewrite).
To deparse the createstmt we need to access the catalog info about the newly
created table, so I invoke this after defining table.
> 4. It should be published for all tables publication similar to 'create table'
Agreed.
> 5. The new code in CreatePublication looks quite haphazard, can we improve it
> by moving it into separate functions?
Changed.
Thanks for the comments.
Here is the new version POC patch set for DDL replication.
For V7-0002, I tried to improve the code in publicationcmds.c by introducing a
common function to create the event trigger which can avoid some duplicate
code. Besides, I merged Ajin's DROP CASCADE patch into it. Also adjusted some
other code style.
For V7-0003, I changed the new type trigger to only catch the CREATE TABLE
AS/SELECT INTO. And rename it to "table_init_write" trigger which seems
consistent for its usage. I kept the function
EventTriggerTableInitWriteStart/End as we need these to store the original
parsetree which is needed to filter the tagname(We need to use the tag in
original parsetree to judge which command should fire the event trigger, see
function EventTriggerCommonSetup()). Besides, I also adjusted code try to make
it look less hacky.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v7-0002-Support-DDL-replication.patch (128.5K, ../../OS0PR01MB57167C38E7D7FA5388DEDB5294A59@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v7-0002-Support-DDL-replication.patch)
download | inline diff:
From 183c8261397b019a614c4c17146348d73ddf58e4 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 14:00:07 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER TABLE command and CREATE TABLE:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
- For DROP TABLE:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/ddl_deparse.c | 10 +-
src/backend/commands/event_trigger.c | 154 +++++++++-
src/backend/commands/publicationcmds.c | 91 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 187 +++++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 58 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/include/tcop/ddl_deparse.h | 2 +-
src/include/tcop/deparse_utility.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
src/test/subscription/t/032_ddl_replication.pl | 86 ++++++
37 files changed, 1603 insertions(+), 222 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 60c077f..7e20044 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -2069,19 +2069,21 @@ deparse_simple_command(CollectedCommand *cmd)
}
char *
-deparse_drop_table(const char *objidentity)
+deparse_drop_table(const char *objidentity, DropBehavior behavior)
{
StringInfoData str;
char *command;
- ObjTree *stmt;
+ ObjTree *stmt, *stmt2;
char *fmt;
Jsonb *jsonb;
initStringInfo(&str);
- fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
-
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s %%{cascade}s");
stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
jsonb = objtree_to_jsonb(stmt);
command = JsonbToCString(&str, &jsonb->root, 128);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..e5894c1 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,150 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_DropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_Alter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_Alter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_Create;
+ }
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ char *command;
+
+ command = deparse_drop_table(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_DropEnd,
+ command, strlen(command) + 1);
+ }
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..4a596c7 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,40 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1110,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..74108b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5218,7 +5218,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9..233abcd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57..1beda09 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,132 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_DropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_DropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_Alter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_Create:
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2151,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2210,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2324,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2510,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9..7b51fb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924..fe28abb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..bb98c17 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..b80d7ad
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_Create,
+ DCT_DropStart,
+ DCT_DropEnd,
+ DCT_Alter
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..3664873 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ NodeTag cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
index fb174d2..1eb2016 100644
--- a/src/include/tcop/ddl_deparse.h
+++ b/src/include/tcop/ddl_deparse.h
@@ -5,6 +5,6 @@
extern char *deparse_utility_command(CollectedCommand *cmd);
extern char *ddl_deparse_json_to_string(char *jsonb);
-extern char *deparse_drop_table(const char *objidentity);
+extern char *deparse_drop_table(const char *objidentity, DropBehavior behavior);
#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2a38a93..b1921cc 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..c36a7c0
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,86 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test ALTER TABLE command is replicated on table test_rep
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$node_publisher->wait_for_catchup('mysub');
+
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE t4;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(0), 'TABLE t4 is dropped');
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.7.2.windows.1
[application/octet-stream] v7-0003-support-CREATE-TABLE-A-ELECT-INTO.patch (14.1K, ../../OS0PR01MB57167C38E7D7FA5388DEDB5294A59@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v7-0003-support-CREATE-TABLE-A-ELECT-INTO.patch)
download | inline diff:
From e966dcf860b4ea0becd66ce159398a4615b3b3d2 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 6 Jun 2022 20:52:12 +0800
Subject: [PATCH] support CREATE TABLE ASSELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 +++
src/backend/commands/ddl_deparse.c | 18 ++++
src/backend/commands/event_trigger.c | 153 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 207 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 7e20044..fcba4bb 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -2035,6 +2035,21 @@ removed feature
return alterTableStmt;
}
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
/*
* Handle deparsing of simple commands.
*
@@ -2135,6 +2150,9 @@ deparse_utility_command(CollectedCommand *cmd)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index e5894c1..5256e1a 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -586,7 +588,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -869,6 +872,144 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_Create,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1149,7 +1290,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1873,6 +2015,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1890,6 +2033,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 4a596c7..3ac20ce 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -957,6 +957,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_TABLE
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -968,6 +973,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..4d003de 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index 3a9c9f0..76b2315 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bb98c17..b82a5a4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11899,4 +11899,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v7-0001-Functions-to-deparse-DDL-commands.patch (85.8K, ../../OS0PR01MB57167C38E7D7FA5388DEDB5294A59@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v7-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 275544b1a7f7e5d68f22c66ffd21b0c3e2dccc07 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE/ALTER TABLE/DROP TABLE specific
functions. Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 2180 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 +++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 7 +-
src/include/tcop/ddl_deparse.h | 10 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2957 insertions(+), 1 deletion(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..60c077f
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,2180 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..fb174d2
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,10 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-09 11:43 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-09 11:43 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com>
Hi,
I did some research for one potential problem[1] mentioned earlier which is related
to the function execution when replicating DDL.
[1]> 4. Statements that have nondeterministic side effects (e.g., as caused
> by triggers, stored procedures, user-defined functions) may result in
> different side effects occurring on each subscriber.
>
> Think how to handle triggers and functions with same name but different
> purpose.
>
Examples:
ALTER TABLE ADD CONSTRAINT func()
ALTER TABLE ADD COLUMN DEFAULT func()
CREATE TRIGGER ... execute procedure func()
...
When replication the above DDLs, there are some cases we need to think about.
----------------
1) The functions used in DDL have same definition among pub/sub.
In this case, if the functions include DML/DDL operations, it will result in
unexpected behavior.
For example:
--- both pub and sub have the same function test_func().
create function test_func()
returns integer as '
begin
CREATE TABLE test(a int);
INSERT INTO test values(1);
return 0;
end
' language plpgsql;
--- We replicate the following DDL
ALTER TABLE t ADD COLUMN a int DEFAULT test_func();
There are three SQLs that would be replicated to the subscriber:
CREATE TABLE test(a int);
INSERT(1);
ALTER TABLE t ADD COLUMN a int DEFAULT test_func();
Then, we would create the table "test" twice and insert same value twice(first by
executing DEFAULT function, second by replaying the CREATE TABLE and INSERT
command) on subcriber.
One possible idea is that we only allow replicating 'immutable/stable' function
which won't include write action so that we don't have this problem. But it
seems a bit restrictive.
----------------
2) The functions used in DDL have different definitions among pub/sub.
I am not sure how to handle this case as this could result in unpredictable
behavior based on the different definitions. And it's difficult to compare the
function definition among pub/sub because the function could call other
functions nested.
OTOH, the current behavior of this case on HEAD is that we don't check the
consistency of the functions executed on pub/sub. For example: the row triggers
will always be fired on subscriber(when enabled) without checking if the same
trigger exists on publisher. So, it might be acceptable if we document this
restriction, although I am not sure.
*******************
- About the solution for the above two points. I think one solution could be:
We can document that user should make sure the DDL to be replicated should not
execute any function which could execute DML/DDL. This seems acceptable as it's
a pretty rare case to execute DML/DDL in a CONSTRAINT function or DEFAULT
function. And the document[1] already suggest similar thing for CONSTRAINT
function. Besides, we can also document that and the functions should be
defined in a consistent way among pub/sub.
[1] https://www.postgresql.org/docs/devel/ddl-constraints.html
> PostgreSQL assumes that CHECK constraints' conditions are immutable, that is,
> they will always give the same result for the same input row. This assumption
> is what justifies examining CHECK constraints only when rows are inserted or
> updated, and not at other times. (The warning above about not referencing
> other table data is really a special case of this restriction.)
- Another solution could be:
We coud introduce a new function flag called (replica_safety) and the values
could be 'safe'/'unsafe'. 'safe' indicates that it's safe to be replicated to
subscriber and it's safe to be executed when replay the DDL from other
publisher.
When replicating the DDL on publisher, we first check if the functions used in
DDL are replica_safe, if not, we don't replicate that DDL. If yes, before
replicating the DDL, we first send a new type FUNCTIONS message which include
functions(used in DDL or ..) information to the subscriber which is similar to
the RELATION messages. And on subscriber, we check if the received functions
exist and if they are also marked replica safe on subscriber. if functions are
not replica safe on sub, we report an error and suggest user to adjust the
problematic function. I haven't tried to implement it so I could miss
something. Thoughts ?
BTW, attach the POC patch set which only fixed a CFbot error and
added some testcases provided by Osumi-san.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v8-0004-Test-cases-for-DDL-replication.patch (13.2K, ../../OS0PR01MB5716733D6C79D2198E5773CC94A79@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v8-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From e74f2a3ee51134cce35a84ae47fd5e3523f6b685 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@fujitsu.com>
Date: Thu, 9 Jun 2022 18:05:31 +0800
Subject: [PATCH] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 236 +++++++++++++++++++++++++
1 file changed, 236 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..5bf7109
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,236 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table syncrhonization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER COLUMN TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4', '4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, '4', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp4 (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp4 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp4';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp4;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp4");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp5 (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp5 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp5';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp5;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp5");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp6 (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp6 (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp6 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp6;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp6");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp6 (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp6 (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp6 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp6;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp6");
+
+# Test CREATE TABLE with constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp7 (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp7 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp7 VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp7_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp7;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp8 (id int) WITH (fillfactor = 80);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp8';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp8;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace itself is not replicated)
+my $basedir = $node_publisher->basedir;
+my $tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp12 (id int) TABLESPACE mytblspc;");
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp12 VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp12;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.7.2.windows.1
[application/octet-stream] v8-0001-Functions-to-deparse-DDL-commands.patch (85.8K, ../../OS0PR01MB5716733D6C79D2198E5773CC94A79@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v8-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 275544b1a7f7e5d68f22c66ffd21b0c3e2dccc07 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for CREATE TABLE/ALTER TABLE/DROP TABLE specific
functions. Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 2180 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 +++++++++++++
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 7 +-
src/include/tcop/ddl_deparse.h | 10 +
src/include/utils/ruleutils.h | 1 +
7 files changed, 2957 insertions(+), 1 deletion(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..60c077f
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,2180 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "lib/ilist.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..fb174d2
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,10 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
2.7.2.windows.1
[application/octet-stream] v8-0002-Support-DDL-replication.patch (124.7K, ../../OS0PR01MB5716733D6C79D2198E5773CC94A79@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v8-0002-Support-DDL-replication.patch)
download | inline diff:
From 08177896c7658bd74a3122527702dcb78ea0c865 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 9 Jun 2022 17:47:35 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER TABLE command and CREATE TABLE:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
- For DROP TABLE:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/ddl_deparse.c | 10 +-
src/backend/commands/event_trigger.c | 154 +++++++++-
src/backend/commands/publicationcmds.c | 91 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 187 +++++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 58 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/include/tcop/ddl_deparse.h | 2 +-
src/include/tcop/deparse_utility.h | 1 +
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
36 files changed, 1517 insertions(+), 222 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 60c077f..7e20044 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -2069,19 +2069,21 @@ deparse_simple_command(CollectedCommand *cmd)
}
char *
-deparse_drop_table(const char *objidentity)
+deparse_drop_table(const char *objidentity, DropBehavior behavior)
{
StringInfoData str;
char *command;
- ObjTree *stmt;
+ ObjTree *stmt, *stmt2;
char *fmt;
Jsonb *jsonb;
initStringInfo(&str);
- fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s");
-
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s %%{cascade}s");
stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
jsonb = objtree_to_jsonb(stmt);
command = JsonbToCString(&str, &jsonb->root, 128);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..e5894c1 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,150 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_DropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_Alter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_Alter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_Create;
+ }
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ char *command;
+
+ command = deparse_drop_table(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_DropEnd,
+ command, strlen(command) + 1);
+ }
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..4a596c7 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,40 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1110,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..74108b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5218,7 +5218,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9..233abcd 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2445,6 +2449,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2738,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57..1beda09 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,132 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_DropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_DropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_Alter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_Create:
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2151,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2210,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2324,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2510,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9..7b51fb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924..fe28abb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..bb98c17 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..b80d7ad
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,58 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_Create,
+ DCT_DropStart,
+ DCT_DropEnd,
+ DCT_Alter
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..de58035 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
index fb174d2..1eb2016 100644
--- a/src/include/tcop/ddl_deparse.h
+++ b/src/include/tcop/ddl_deparse.h
@@ -5,6 +5,6 @@
extern char *deparse_utility_command(CollectedCommand *cmd);
extern char *ddl_deparse_json_to_string(char *jsonb);
-extern char *deparse_drop_table(const char *objidentity);
+extern char *deparse_drop_table(const char *objidentity, DropBehavior behavior);
#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v8-0003-support-CREATE-TABLE-A-ELECT-INTO.patch (14.1K, ../../OS0PR01MB5716733D6C79D2198E5773CC94A79@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v8-0003-support-CREATE-TABLE-A-ELECT-INTO.patch)
download | inline diff:
From e966dcf860b4ea0becd66ce159398a4615b3b3d2 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Mon, 6 Jun 2022 20:52:12 +0800
Subject: [PATCH] support CREATE TABLE ASSELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 +++
src/backend/commands/ddl_deparse.c | 18 ++++
src/backend/commands/event_trigger.c | 153 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 207 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 7e20044..fcba4bb 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -2035,6 +2035,21 @@ removed feature
return alterTableStmt;
}
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
/*
* Handle deparsing of simple commands.
*
@@ -2135,6 +2150,9 @@ deparse_utility_command(CollectedCommand *cmd)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index e5894c1..5256e1a 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -586,7 +588,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -869,6 +872,144 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_Create,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1149,7 +1290,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1873,6 +2015,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1890,6 +2033,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 4a596c7..3ac20ce 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -957,6 +957,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_TABLE
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -968,6 +973,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..4d003de 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index 3a9c9f0..76b2315 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bb98c17..b82a5a4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11899,4 +11899,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-10 06:26 Amit Kapila <amit.kapila16@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-06-10 06:26 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Zheng Li <zhengli10@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; osumi.takamichi@fujitsu.com <osumi.takamichi@fujitsu.com>
On Thu, Jun 9, 2022 at 5:14 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> Hi,
>
> I did some research for one potential problem[1] mentioned earlier which is related
> to the function execution when replicating DDL.
>
> [1]> 4. Statements that have nondeterministic side effects (e.g., as caused
> > by triggers, stored procedures, user-defined functions) may result in
> > different side effects occurring on each subscriber.
> >
> > Think how to handle triggers and functions with same name but different
> > purpose.
> >
>
> Examples:
> ALTER TABLE ADD CONSTRAINT func()
> ALTER TABLE ADD COLUMN DEFAULT func()
> CREATE TRIGGER ... execute procedure func()
> ...
>
> When replication the above DDLs, there are some cases we need to think about.
>
> ----------------
> 1) The functions used in DDL have same definition among pub/sub.
>
> In this case, if the functions include DML/DDL operations, it will result in
> unexpected behavior.
>
> For example:
>
> --- both pub and sub have the same function test_func().
> create function test_func()
> returns integer as '
> begin
> CREATE TABLE test(a int);
> INSERT INTO test values(1);
> return 0;
> end
> ' language plpgsql;
>
> --- We replicate the following DDL
> ALTER TABLE t ADD COLUMN a int DEFAULT test_func();
>
> There are three SQLs that would be replicated to the subscriber:
> CREATE TABLE test(a int);
> INSERT(1);
> ALTER TABLE t ADD COLUMN a int DEFAULT test_func();
>
> Then, we would create the table "test" twice and insert same value twice(first by
> executing DEFAULT function, second by replaying the CREATE TABLE and INSERT
> command) on subcriber.
>
The other kind of problem is that we don't know whether the tables
being accessed by these DDL/DML are published or not. So blindly
allowing such functions can allow unintended clauses like the tables
accessed in those functions may not even exist on the subscriber-side.
> One possible idea is that we only allow replicating 'immutable/stable' function
> which won't include write action so that we don't have this problem. But it
> seems a bit restrictive.
>
> ----------------
>
> 2) The functions used in DDL have different definitions among pub/sub.
>
> I am not sure how to handle this case as this could result in unpredictable
> behavior based on the different definitions. And it's difficult to compare the
> function definition among pub/sub because the function could call other
> functions nested.
>
> OTOH, the current behavior of this case on HEAD is that we don't check the
> consistency of the functions executed on pub/sub. For example: the row triggers
> will always be fired on subscriber(when enabled) without checking if the same
> trigger exists on publisher. So, it might be acceptable if we document this
> restriction, although I am not sure.
>
> *******************
>
> - About the solution for the above two points. I think one solution could be:
>
> We can document that user should make sure the DDL to be replicated should not
> execute any function which could execute DML/DDL. This seems acceptable as it's
> a pretty rare case to execute DML/DDL in a CONSTRAINT function or DEFAULT
> function. And the document[1] already suggest similar thing for CONSTRAINT
> function. Besides, we can also document that and the functions should be
> defined in a consistent way among pub/sub.
>
> [1] https://www.postgresql.org/docs/devel/ddl-constraints.html
> > PostgreSQL assumes that CHECK constraints' conditions are immutable, that is,
> > they will always give the same result for the same input row. This assumption
> > is what justifies examining CHECK constraints only when rows are inserted or
> > updated, and not at other times. (The warning above about not referencing
> > other table data is really a special case of this restriction.)
>
> - Another solution could be:
>
> We coud introduce a new function flag called (replica_safety) and the values
> could be 'safe'/'unsafe'. 'safe' indicates that it's safe to be replicated to
> subscriber and it's safe to be executed when replay the DDL from other
> publisher.
>
Yeah, something like this could be a good idea but I think for the
first version we should simply raise a WARNING for such DDLs
indicating that they won't be replicated.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-12 06:46 Zheng Li <zhengli10@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-06-12 06:46 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> > > > > I've not looked at these patches in-depth yet but with this approach,
> > > > > what do you think we can handle the DDL syntax differences between
> > > > > major versions? DDL syntax or behavior could be changed by future
> > > > > changes and I think we need to somehow deal with the differences. For
> > > >
> > > > > example, if the user uses logical replication for major version
> > > > > upgrade, the publisher is older than the subscriber. We might have to
> > > > > rewrite the DDL before applying to the subscriber because the DDL
> > > > > executed on the publisher no longer work on a new PostgreSQL version
> > > >
> > > > I don't think we will allow this kind of situation to happen in the
> > > > first place for
> > > > backward compatibility. If a DDL no longer works on a new version of
> > > > PostgreSQL, the user will have to change the application code as well.
> > > > So even if it happens for
> > > > whatever reason, we could either
> > > > 1. fail the apply worker and let the user fix such DDL because they'll
> > > > have to fix the application code anyway when this happens.
> > > > 2. add guard rail logic in the apply worker to automatically fix such
> > > > DDL if possible, knowing the version of the source and target. Similar
> > > > logic must have been implemented for pg_dump/restore/upgrade.
> > > >
> > > > > or we might have to add some options to the DDL before the application
> > > > > in order to keep the same behavior. This seems to require a different
> > > > > solution from what the patch does for the problem you mentioned such
> > > >
> > > > > as "DDL involving multiple tables where only some tables are
> > > > > replicated”.
> > > >
> > > > First of all, this case can only happen when the customer chooses to
> > > > only replicate a subset of the tables in a database in which case
> > > > table level DDL replication is chosen instead of database level DDL
> > > > replication (where all tables
> > > > and DDLs are replicated). I think the solution would be:
> > > > 1. make best effort to detect such DDLs on the publisher and avoid
> > > > logging of such DDLs in table level DDL replication.
> > > > 2. apply worker will fail to replay such command due to missing
> > > > objects if such DDLs didn't get filtered on the publisher for some
> > > > reason. This should be rare and I think it's OK even if it happens,
> > > > we'll find out
> > > > why and fix it.
> > > >
> > >
> > > FWIW, both these cases could be handled with the deparsing approach,
> > > and the handling related to the drop of multiple tables where only a
> > > few are published is already done in the last POC patch shared by Ajin
> > > [1].
> > >
> >
> > Right. So I'm inclined to think that deparsing approach is better from
> > this point as well as the point mentioned by Álvaro before[1].
>
> I agree. One more point about deparsing approach is that it can also
> help to replicate CREATE TABLE AS/SELECT INTO in a better way.
>
> The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
> TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
> after creating the table and before writing data into the table and replicate
> the incoming writes later as normal INSERTs. In this apporach, we don't execute
> the subquery on subscriber so that don't need to make sure all the objects
> referenced in the subquery also exists in subscriber. And This approach works
> for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
>
> One problem of this approach is that we cannot use the current trigger to
> deparse or WAL log the CREATE TABLE. Because none of the even trigger is fired
> after creating the table and before inserting the data. To solve this, one idea
> is that we could directly add some code at the end of create_ctas_internal() to
> deparse and WAL log it. Moreover, we could even introduce a new type of event
> trigger(table_create) which would be fired at the expected timing so that we
> can use the trigger function to deparse and WAL log. I am not sure which way is
> better. I temporarily use the second idea which introduce a new type event
> trigger in the 0003 POC patch.
Hi, I agree that an intermediate structured format (with all objects
schema qualified) makes it easier to handle syntax differences between
the publisher and the subscriber. Such structured format is also
likely easier to use by other logical replication consumers.
However, to make things more maintainable, would it be better to use
the existing serialization/deserialization functions in
out/readfuncs.c to generate the parsetree representation of the DDL
command?
It turns out support for DDL commands are mostly missing in
out/readfuncs at the moment. see the following comment from
outfuncs.c:
* Output support for raw parsetrees
* is somewhat incomplete, too; in particular, utility statements are
* almost entirely unsupported. We try to support everything that can
* appear in a raw SELECT, though.
So what about adding support for utility statements in out/readfuncs.c
so that nodeToString()/stringToNode() works on raw parsetrees of
utility statements? I think there'll be some benefits:
1. It's less code compared to introducing brand new
serialization/deserialization code for DDL commands.
2. It's more maintainable since hackers are already familiar with
out/readfuncs.c.
3. Support for utility statements in out/readfuncs.c may turn out to
be useful for future features that need serialization/deserialization
of DDL raw parsetrees.
In patch 0012 I explored the above idea by changing the logging format
of logical ddlmessage to that of the serialization of the raw
parsetree of the DDL command using nodeToString(), the apply worker
deserializes the message back to the raw parsetree by calling
stringToNode().
Here are the detailed changes:
- Adding serialization/deserialization functions in outfuncs.c/readfuncs.c
for CreateTableStmt, AlterTableStmt, DropStmt, CreateFunctionStmt and
AlterFunctionStmt.
- Modified the serialization process to always schema qualify object names,
this is done by outQualifiedName() and a change in _outRangeVar().
- Change the input of LogLogicalDDLMessage() to use nodeToString(parsetree).
- Change the apply worker to call stringToNode(ddlmessage) to get
the raw parsetree back and then directly execute the parsetree.
This patch doesn't introduce any deparsing functionality yet, but it
also provides the flexibility to edit the command being replayed by
directly modifying the raw parsetree on the apply worker (e.g. setting
the
missing_ok flag for DropStmt is equivalent to adding "IF EXIST" to the
statement).
Thoughts?
Regards,
Zheng Li
Attachments:
[application/octet-stream] 0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch (4.1K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/2-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch)
download | inline diff:
From 9fae6f77eed1879befabd0cddce24065af78a170 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 22 Mar 2022 23:47:52 +0000
Subject: [PATCH 04/12] Enable replication of CREATE MATERIALIZED VIEW AS stmt.
Add tests for both CREATE MATERIALIZED VIEW and CREATE VIEW.
---
src/backend/tcop/utility.c | 42 +++++++++++++++++++++++--
src/test/subscription/t/030_rep_ddls.pl | 18 +++++++++++
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index d0bb232af6..79b5faccc0 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1140,6 +1140,45 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
}
break;
+ /*
+ * CreateTableAsStmt can create either a table a materialized view
+ * and they are handled differently.
+ */
+ case T_CreateTableAsStmt:
+ {
+ CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+ switch(stmt->objtype)
+ {
+ case OBJECT_TABLE:
+ /*
+ * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
+ * 1. A normal CREATE TABLE string that get's logged and replicated via
+ * DDL replication.
+ * 2. Insertions that get replicated by DML replication.
+ */
+ break;
+ case OBJECT_MATVIEW:
+ /*
+ * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ default:
+ break;
+ }
+ break;
+ }
+
/*
* Secondly, commands that may be allowed in Table level DDL replication.
* These are currently handled in the later execution path of the command.
@@ -1155,7 +1194,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/* DropStmt depends on the removeType */
case T_DropStmt:
{
- DropStmt* stmt = (DropStmt*) parsetree;
+ DropStmt *stmt = (DropStmt *) parsetree;
switch (stmt->removeType)
{
/* Maybe allowed in Table level DDL replication, handled in later code path */
@@ -1206,7 +1245,6 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
case T_RuleStmt:
case T_CreateSeqStmt:
case T_AlterSeqStmt:
- case T_CreateTableAsStmt:
case T_RefreshMatViewStmt:
case T_CreatePLangStmt:
case T_CreateConversionStmt:
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index c88c4ea1c0..562efe2cf7 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -226,6 +226,24 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+# Test CREATE MATERIALIZED VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE MATERIALIZED VIEW s1.matview1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+is($result, qq($result_sub), 'CREATE of s1.matview1 is replicated');
+
+# Test CREATE VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.view1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch (8.0K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/3-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch)
download | inline diff:
From aeb5f5c4517f5fcda1c4786ebcd92343fea68f95 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Thu, 24 Mar 2022 17:46:55 +0000
Subject: [PATCH 05/12] Support replication of CREATE ... AS .. and SELECT ...
INTO ... statements.
The idea is to force skipping the direct data population
(which could potentially cause data mismatch compared to the publisher)
with these command on the subscriber by force setting the skipData flag
in the intoClause of the parsetree after the logical replication worker
parses the command. The data sync will be taken care of by DML replication
after the replication of the DDL command.
---
src/backend/replication/logical/worker.c | 51 ++++++++++++++++++++----
src/backend/tcop/utility.c | 41 +++++++++++--------
src/test/subscription/t/030_rep_ddls.pl | 24 +++++++++++
3 files changed, 92 insertions(+), 24 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index c25e3a4cfe..905f47faf5 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2542,23 +2542,58 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
Portal portal;
DestReceiver *receiver;
bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE stmt only */
- char *relname = NULL; /* For CREATE TABLE stmt only */
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
commandTag = CreateCommandTag((Node *)command);
/*
- * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
- * execute the stmt. At that point, CreateStmt may have been freeed up.
+ * execute the stmt. At that point, command->stmt may have been freeed up.
*/
if (commandTag == CMDTAG_CREATE_TABLE)
{
- CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
}
+ else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ RangeVar *rv = castmt->into->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ /* SELECT INTO */
+ else if (commandTag == CMDTAG_SELECT)
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ RangeVar *rv = sstmt->intoClause->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
@@ -2657,12 +2692,12 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
* Table created by DDL replication (database level) is automatically
* added to the subscription here.
*
- * Call AddSubscriptionRelState for CREATE TABEL command to set
- * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+ * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
* new table can be replicated without having to manually run
* "alter subscription ... refresh publication"
*/
- if (commandTag == CMDTAG_CREATE_TABLE)
+ if (relname != NULL)
{
Oid relid;
Oid relnamespace = InvalidOid;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 79b5faccc0..29381b53b6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1142,24 +1142,31 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/*
* CreateTableAsStmt can create either a table a materialized view
- * and they are handled differently.
*/
case T_CreateTableAsStmt:
{
CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+
switch(stmt->objtype)
{
+ /*
+ * Either CREATE TABLE AS or SELECT ... INTO stmt
+ * The statement is logged as is, but when we apply the
+ * CREATE TABLE AS or SELECT ... INTO statemtns on the logical
+ * replication worker, we will force the skipData flag in the
+ * intoClause. This way we avoid direct data population on the
+ * subsriber with the execution of these commands which can
+ * potentially cause data mismatch bewteen the publisher.
+ *
+ * The data sync will be handled by DML replication after the
+ * target table has been created.
+ */
case OBJECT_TABLE:
- /*
- * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
- * 1. A normal CREATE TABLE string that get's logged and replicated via
- * DDL replication.
- * 2. Insertions that get replicated by DML replication.
- */
- break;
+
+ /* CREATE MATERIALIZED VIEW */
case OBJECT_MATVIEW:
/*
- * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * Log these stmt for logical replication if
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
@@ -1304,8 +1311,17 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
if (isCompleteQuery)
+ {
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+ }
+
switch (nodeTag(parsetree))
{
/*
@@ -2141,13 +2157,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a complete top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 562efe2cf7..3b15c6d9f0 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -244,6 +244,30 @@ $node_publisher->wait_for_catchup('mysub');
$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+# TEST CREATE TABLE AS stmt
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t3 AS SELECT a, b from s1.t1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t3;");
+is($result, qq(2), 'CREATE TABLE s1.t3 AS is replicated with data');
+
+# TEST CREATE TABLE AS stmt ... WITH NO DATA
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t4 AS SELECT a, b from s1.t1 WITH NO DATA;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t4;");
+is($result, qq(0), 'CREATE TABLE s1.t4 AS is replicated with no data');
+
+# TEST SELECT INTO stmt
+$node_publisher->safe_psql('postgres', "SELECT b into s1.t6 from s1.t1 where a > 1");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
+is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch (65.0K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/4-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch)
download | inline diff:
From ff8fcd9b1a5e4b664977a1ced1fa76c95b76199d Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:47:16 +0000
Subject: [PATCH 01/12] Define DDL replication levels via the CREATE
PUBLICATION command, currently allow Database or Table level DDL replication.
Allow the user to configure either database level or table level
DDL replication via the CREATE PUBLICATION command as in my first
email. Two new columns are added to the pg_publication catalog to
show the DDL replication levels, test output publication.out is
updated accordingly. pg_dump is also modified to accommodate the
pg_publication catalog change.
---
src/backend/catalog/pg_publication.c | 2 +
src/backend/commands/publicationcmds.c | 89 +++++-
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/pg_dump.c | 45 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +-
src/bin/psql/describe.c | 30 +-
src/include/catalog/pg_publication.h | 10 +
src/include/commands/defrem.h | 1 +
src/test/regress/expected/publication.out | 372 +++++++++++-----------
10 files changed, 364 insertions(+), 199 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca62de..b8ab1d8141 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1006,6 +1006,8 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubactions.pubddl_database = pubform->pubddl_database;
+ pub->pubactions.pubddl_table = pubform->pubddl_table;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e645741e4..edceaa0c4e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -80,15 +80,18 @@ static void PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok);
static void
parse_publication_options(ParseState *pstate,
List *options,
+ bool for_all_tables,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *ddl_level_given)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *ddl_level_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -96,6 +99,16 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ if (for_all_tables)
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ else
+ {
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+ }
/* Parse options */
foreach(lc, options)
@@ -154,6 +167,55 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "ddl") == 0)
+ {
+ char *ddl_level;
+ List *ddl_list;
+ ListCell *lc;
+
+ if (*ddl_level_given)
+ errorConflictingDefElem(defel, pstate);
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
+ ddl_level = defGetString(defel);
+
+ if (!SplitIdentifierString(ddl_level, ',', &ddl_list))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax for \"ddl\" option")));
+
+ /* Process the option list. */
+ foreach(lc, ddl_list)
+ {
+ char *publish_opt = (char *) lfirst(lc);
+
+ if (strcmp(publish_opt, "database") == 0)
+ {
+ if (!for_all_tables)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("DDL replication on the database level is only supported in FOR ALL TABLES publication")));
+ else
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ }
+ else if (strcmp(publish_opt, "table") == 0)
+ pubactions->pubddl_table = true;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized \"ddl\" value: \"%s\"", publish_opt)));
+ }
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -761,6 +823,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Datum values[Natts_pg_publication];
HeapTuple tup;
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -803,9 +866,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
parse_publication_options(pstate,
stmt->options,
+ stmt->for_all_tables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -822,6 +887,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubddl_database - 1] =
+ BoolGetDatum(pubactions.pubddl_database);
+ values[Anum_pg_publication_pubddl_table - 1] =
+ BoolGetDatum(pubactions.pubddl_table);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -908,6 +977,7 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
bool replaces[Natts_pg_publication];
Datum values[Natts_pg_publication];
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -916,11 +986,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
List *root_relids = NIL;
ListCell *lc;
+ pubform = (Form_pg_publication) GETSTRUCT(tup);
+
parse_publication_options(pstate,
stmt->options,
+ pubform->puballtables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -1024,6 +1098,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
+ if (ddl_level_given)
+ {
+ values[Anum_pg_publication_pubddl_database - 1] = BoolGetDatum(pubactions.pubddl_database);
+ replaces[Anum_pg_publication_pubddl_database - 1] = true;
+
+ values[Anum_pg_publication_pubddl_table - 1] = BoolGetDatum(pubactions.pubddl_table);
+ replaces[Anum_pg_publication_pubddl_table - 1] = true;
+ }
+
if (publish_via_partition_root_given)
{
values[Anum_pg_publication_pubviaroot - 1] = BoolGetDatum(publish_via_partition_root);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 60e72f9e8b..dd235d66d0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5633,6 +5633,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl_database |= pubform->pubddl_database;
+ pubdesc->pubactions.pubddl_table |= pubform->pubddl_table;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72e49..8860510ac1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3868,6 +3868,8 @@ getPublications(Archive *fout, int *numPublications)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubddl_database;
+ int i_pubddl_table;
int i,
ntups;
@@ -3882,23 +3884,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubddl_database, p.pubddl_table "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3915,6 +3923,8 @@ getPublications(Archive *fout, int *numPublications)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubddl_database = PQfnumber(res, "pubddl_database");
+ i_pubddl_table = PQfnumber(res, "pubddl_table");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3939,6 +3949,10 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubddl_database =
+ (strcmp(PQgetvalue(res, i, i_pubddl_database), "t") == 0);
+ pubinfo[i].pubddl_table =
+ (strcmp(PQgetvalue(res, i, i_pubddl_table), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4018,6 +4032,29 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
appendPQExpBufferStr(query, "'");
+ appendPQExpBufferStr(query, ", ddl = '");
+ first = true;
+
+ if (pubinfo->pubddl_database)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "database");
+ first = false;
+ }
+
+ if (pubinfo->pubddl_table)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "table");
+ first = false;
+ }
+
+ appendPQExpBufferStr(query, "'");
+
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..8c5ef01cb0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,8 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1f08716f69..cbfcabcfa8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2420,7 +2420,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2429,9 +2429,9 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub2
FOR ALL TABLES
- WITH (publish = \'\');',
+ WITH (publish = \'\', ddl = \'\');',
regexp => qr/^
- \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '');\E
+ \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2440,7 +2440,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub3;',
regexp => qr/^
- \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2449,7 +2449,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub4;',
regexp => qr/^
- \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924a23..b07c6b1841 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,15 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_database AS \"%s\"",
+ gettext_noop("Database level DDLs"));
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_table AS \"%s\"",
+ gettext_noop("Table level DDLs"));
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6181,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6198,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6212,11 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ {
+ appendPQExpBufferStr(&buf,
+ ", pubddl_database, pubddl_table");
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6265,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols += 2;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6281,11 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ {
+ printTableAddHeader(&cont, gettext_noop("Database level DDL"), true, align);
+ printTableAddHeader(&cont, gettext_noop("Table level DDL"), true, align);
+ }
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6296,11 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ {
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
+ printTableAddCell(&cont, PQgetvalue(res, i, 10), false, false);
+ }
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba429..a94e152256 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -20,6 +20,7 @@
#include "catalog/genbki.h"
#include "catalog/objectaddress.h"
#include "catalog/pg_publication_d.h"
+#include "tcop/utility.h"
/* ----------------
* pg_publication definition. cpp turns this into
@@ -54,6 +55,12 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if database level ddls are published */
+ bool pubddl_database;
+
+ /* true if table level ddls are published */
+ bool pubddl_table;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +79,8 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationActions;
typedef struct PublicationDesc
@@ -157,5 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..073f46ef1a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -15,6 +15,7 @@
#define DEFREM_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_publication.h"
#include "nodes/params.h"
#include "parser/parse_node.h"
#include "tcop/dest.h"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37dfe5..e9cea11a5b 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | t | t | t | f | f | f | t | t
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.32.0
[application/octet-stream] 0003-Logical-replication-of-DDL-messages.patch (41.2K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/5-0003-Logical-replication-of-DDL-messages.patch)
download | inline diff:
From 4d4f72483c180730cc89854baaf198f08e44fd45 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 17:17:39 +0000
Subject: [PATCH 03/12] Logical replication of DDL messages
Integration with pgoutput:
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced
for this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
Logical replication worker change:
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to SUBREL_STATE_INIT,
So that DML replication can progress on this new table without
manually running "ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
TAP test:
A new TAP test 030_rep_ddl.pl is added. We mainly focused on
testing the happy path of database level replication so far.
Corner case DDLs and table level DDL replication are still to be
carefully tested.
---
.../test_decoding/expected/ddlmessages.out | 26 +-
contrib/test_decoding/sql/ddlmessages.sql | 5 +-
src/backend/replication/logical/proto.c | 63 ++++-
src/backend/replication/logical/worker.c | 264 ++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 82 +++++-
src/backend/tcop/utility.c | 14 +-
src/include/replication/logicalproto.h | 10 +-
src/test/subscription/t/004_sync.pl | 2 +-
src/test/subscription/t/006_rewrite.pl | 2 +-
src/test/subscription/t/008_diff_schema.pl | 2 +-
src/test/subscription/t/009_matviews.pl | 2 +-
src/test/subscription/t/012_collation.pl | 2 +-
src/test/subscription/t/013_partition.pl | 8 +-
src/test/subscription/t/030_rep_ddls.pl | 237 ++++++++++++++++
14 files changed, 684 insertions(+), 35 deletions(-)
create mode 100644 src/test/subscription/t/030_rep_ddls.pl
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 79284f9def..0376f36c24 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -2,6 +2,9 @@
SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
?column?
----------
@@ -20,23 +23,22 @@ BEGIN;
CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
-SELECT pg_drop_replication_slot('regression_slot');
-DROP TABLE test_ddlmessage;
-DROP publication mypub;
- data
---------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
(1 row)
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index 211497ee22..c23610f9b4 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -3,6 +3,10 @@ SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
+
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
CREATE TABLE test_ddlmessage (id serial unique, data int);
@@ -20,7 +24,6 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e2d2..3cfa94dd8c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -662,6 +662,63 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
pq_sendbytes(out, message, sz);
}
+/*
+ * Read DDL MESSAGE from stream
+ */
+const char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ const char **role,
+ const char **search_path,
+ bool *transactional,
+ Size *sz)
+{
+ uint8 flags;
+ const char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *role = pq_getmsgstring(in);
+ *search_path = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* encode and send message flags */
+ if (transactional)
+ flags |= MESSAGE_TRANSACTIONAL;
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendstring(out, role);
+ pq_sendstring(out, search_path);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
/*
* Write relation description to the output stream.
*/
@@ -1218,6 +1275,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index fc210a9e7b..c25e3a4cfe 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -180,6 +181,8 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "tcop/pquery.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -346,6 +349,10 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
+static void apply_execute_sql_command(const char *cmdstr,
+ const char* role,
+ const char* search_path,
+ bool isTopLevel);
/* Compute GID for two_phase transactions */
static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
@@ -2445,6 +2452,259 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/*
+ * Handle generic messages.
+ */
+static void
+apply_handle_ddlmessage(StringInfo s)
+{
+ XLogRecPtr lsn;
+ bool transactional;
+ Size sz;
+ const char *prefix;
+ const char *role;
+ const char *search_path;
+ const char *msg;
+
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+
+ apply_execute_sql_command(msg, role, search_path, true);
+}
+
+/*
+ * Add context to the errors produced by apply_execute_sql_command().
+ */
+static void
+execute_sql_command_error_cb(void *arg)
+{
+ errcontext("during execution of SQL statement: %s", (char *) arg);
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE stmt only */
+ char *relname = NULL; /* For CREATE TABLE stmt only */
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ /*
+ * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, CreateStmt may have been freeed up.
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ RangeVar *rv = cstmt->relation;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+ }
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * OK to analyze, rewrite, and plan this query.
+ *
+ * Switch to appropriate context for constructing query and plan trees
+ * (these can't be in the transaction context, as that will get reset
+ * when the command is COMMIT/ROLLBACK). If we have multiple
+ * parsetrees, we use a separate context for each one, so that we can
+ * free that memory before moving on to the next one. But for the
+ * last (or only) parsetree, just use MessageContext, which will be
+ * reset shortly after completion anyway. In event of an error, the
+ * per_parsetree_context will be deleted when MessageContext is reset.
+ */
+ if (lnext(parsetree_list, parsetree_item) != NULL)
+ {
+ per_parsetree_context =
+ AllocSetContextCreate(MessageContext,
+ "per-parsetree message context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+ }
+ else
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(
+ command,
+ cmdstr,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(
+ querytree_list, cmdstr, 0, NULL);
+
+ /*
+ * Done with the snapshot used for parsing/planning.
+ *
+ * While it looks promising to reuse the same snapshot for query
+ * execution (at least for simple protocol), unfortunately it causes
+ * execution to use a snapshot that has been acquired before locking
+ * any of the tables mentioned in the query. This creates user-
+ * visible anomalies, so refrain. Refer to
+ * https://postgr.es/m/flat/5075D8DF.6050500@fuzzy.cz for details.
+ */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in MessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ cmdstr,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ isTopLevel,
+ true,
+ receiver,
+ receiver,
+ NULL);
+ (*receiver->rDestroy) (receiver);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set
+ * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * new table can be replicated without having to manually run
+ * "alter subscription ... refresh publication"
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+ }
+
+ /*
+ * Restore the GUC variables we set above.
+ */
+ AtEOXact_GUC(true, save_nestlevel);
+
+ /* protect against stack resets during CONCURRENTLY processing */
+ if (error_context_stack == &errcallback)
+ error_context_stack = errcallback.previous;
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
/*
* Logical replication protocol message dispatcher.
@@ -2510,6 +2770,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddlmessage(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57143..db221c7e8d 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +260,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +277,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1656,8 +1662,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1703,70 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char * role,
+ const char *search_path, Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+ ListCell *lc;
+
+ /* Reload publications if needed before use. */
+ if (!publications_valid)
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ if (data->publications)
+ list_free_deep(data->publications);
+
+ data->publications = LoadPublications(data->publication_names);
+ MemoryContextSwitchTo(oldctx);
+ publications_valid = true;
+ }
+
+ /* Check if ddl replication is turned on for the publications */
+ foreach(lc, data->publications)
+ {
+ Publication *pub = (Publication *) lfirst(lc);
+ /* TODO need to check relid for table level DDLs */
+ if (!pub->pubactions.pubddl_database && !pub->pubactions.pubddl_table)
+ return;
+ }
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * Output BEGIN if we haven't yet. Avoid for non-transactional
+ * messages.
+ */
+ if (transactional)
+ {
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+ }
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ transactional,
+ prefix,
+ role,
+ search_path,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2052,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl_database =
+ entry->pubactions.pubddl_table = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2112,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl_database = false;
+ entry->pubactions.pubddl_table = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2227,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl_database |= pub->pubactions.pubddl_database;
+ entry->pubactions.pubddl_table |= pub->pubactions.pubddl_table;
/*
* We want to publish the changes as the top-most ancestor
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 5fe54f742a..d0bb232af6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1268,13 +1268,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
switch (nodeTag(parsetree))
{
/*
@@ -2110,6 +2103,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8ff3..28ff562d62 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,14 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
+extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
+ const char **role, const char **search_path,
+ bool *transactional, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index cf61fc1e0f..698c5114e6 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -33,7 +33,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index c924ff35f7..6c9b055eb0 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -23,7 +23,7 @@ $node_subscriber->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index 67b4026afa..2902e6fc34 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -32,7 +32,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 1ce696d4a4..6c586877a1 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -19,7 +19,7 @@ $node_subscriber->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/012_collation.pl b/src/test/subscription/t/012_collation.pl
index 2182f7948e..e1b5050663 100644
--- a/src/test/subscription/t/012_collation.pl
+++ b/src/test/subscription/t/012_collation.pl
@@ -76,7 +76,7 @@ $node_subscriber->safe_psql('postgres',
# set up publication, subscription
$node_publisher->safe_psql('postgres',
- q{CREATE PUBLICATION pub1 FOR ALL TABLES});
+ q{CREATE PUBLICATION pub1 FOR ALL TABLES WITH (ddl = '')});
$node_subscriber->safe_psql('postgres',
qq{CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = false)}
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index e7f4a94f19..007a15dfdc 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -25,9 +25,9 @@ $node_subscriber2->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
# publisher
-$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1");
+$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1 WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_all FOR ALL TABLES");
+ "CREATE PUBLICATION pub_all FOR ALL TABLES WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
"CREATE TABLE tab1 (a int PRIMARY KEY, b text) PARTITION BY LIST (a)");
$node_publisher->safe_psql('postgres',
@@ -425,12 +425,12 @@ $node_publisher->safe_psql('postgres',
# and child tables are present but changes will be replicated via the parent's
# identity and only once.
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# for tab4, we publish changes through the "middle" partitioned table
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# prepare data for the initial sync
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
new file mode 100644
index 0000000000..c88c4ea1c0
--- /dev/null
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -0,0 +1,237 @@
+
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $node_subscriber2 = PostgreSQL::Test::Cluster->new('subscriber2');
+$node_subscriber2->init(allows_streaming => 'logical');
+$node_subscriber2->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber2->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+$node_subscriber2->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+# mypub has pubddl_database on
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+# mypub2 has pubddl_database off
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub2 FOR ALL TABLES with (ddl = '');");
+$node_subscriber2->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub2 CONNECTION '$publisher_connstr' PUBLICATION mypub2;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test smae simple CREATE TABLE command is not replicated to subscriber2 (ddl off)
+# Test ALTER TABLE command is replicated on table test_rep
+# Test CREATE INDEX is replicated to subscriber
+# Test CREATE FUNCTION command is replicated to subscriber
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+$node_publisher->safe_psql('postgres', "CREATE INDEX nameindex on test_rep (name)");
+$node_publisher->safe_psql('postgres', qq{CREATE OR REPLACE FUNCTION totalRecords()
+RETURNS integer AS \$total\$
+declare
+ total integer;
+BEGIN
+ SELECT count(*) into total FROM test_rep;
+ RETURN total;
+END;
+\$total\$ LANGUAGE plpgsql;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't1';");
+is($result, qq(0), 'CREATE of t1 is not replicated to subscriber2');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(1), 'CREATE INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT totalRecords();");
+is($result, qq(2), 'CREATE of function totalRecords replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'CREATE FUNCTION totalrecords is not replicated to subscriber2');
+
+# Test ALTER TABLE DROP
+# Test DROP INDEX
+# Test DROP FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->safe_psql('postgres', "DROP INDEX nameindex;");
+$node_publisher->safe_psql('postgres', "DROP FUNCTION totalRecords;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(0), 'DROP INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'DROP FUNCTION totalrecords replicated');
+
+
+# TODO figure out how to set ON_ERROR_STOP = 0 in this test
+# Test failed CREATE/ALTER TABLE on publisher doesn't break replication
+# Table t1 already exits so expect the command to fail
+#$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+#$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (103, 'data103', 1013);");
+
+#$node_publisher->wait_for_catchup('mysub');
+# Verify replication still works
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+#is($result, qq(1), 'DELETE from test_rep replicated');
+
+# Test DDLs inside txn block
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE t2 (a int, b varchar);
+ALTER TABLE test_rep ADD c3 int;
+INSERT INTO test_rep VALUES (3, 'data3', 3);
+COMMIT;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t2;");
+is($result, qq(0), 'CREATE t2 replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep;");
+is($result, qq(2), 'ALTER test_rep ADD replicated');
+
+# Test toggling pubddl_database option off
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = '');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(f|f), 'pubddl_database turned off on mypub');
+$node_publisher->safe_psql('postgres', "CREATE TABLE t3 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't3';");
+is($result, qq(0), 'CREATE t3 is not replicated');
+
+# Test toggling pubddl_database option on
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = 'database');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(t|t), 'pubddl_database turned on on mypub');
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# A somewhat complicated test in plpgsql block with trigger
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE foo (a int);
+CREATE INDEX foo_idx ON foo (a);
+ALTER TABLE foo ADD COLUMN b timestamptz;
+CREATE FUNCTION foo_ts()
+RETURNS trigger AS $$
+BEGIN
+NEW.b := current_timestamp;
+RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
+FOR EACH ROW EXECUTE FUNCTION foo_ts();
+INSERT INTO foo VALUES (1);
+COMMIT;});
+$result = $node_publisher->safe_psql('postgres', "SELECT b from foo where a = 1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result_sub = $node_subscriber->safe_psql('postgres', "SELECT b from foo where a = 1;");
+is($result, qq($result_sub), 'timestamp of insert matches');
+
+# Test CREATE SCHEMA stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE SCHEMA s1");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_namespace WHERE nspname = 's1';");
+is($result, qq(1), 'CREATE SCHEMA s1 is replicated');
+
+# Test CREATE TABLE in new schema s1 followed by insert
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (2, 'b');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t1;");
+is($result, qq(2), 'CREATE TABLE s1.t1 is replicated');
+
+# Test replication works as expected with mismatched search_path on publisher and subscriber
+$node_publisher->append_conf('postgresql.conf', 'search_path = \'s1, public\'');
+$node_publisher->restart;
+# CREATE unqualified table t2, it is s1.t2 under the modified search_path
+$node_publisher->safe_psql('postgres', "CREATE TABLE t2 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (2, 'b');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (3, 'c');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t2;");
+is($result, qq(3), 'CREATE TABLE s1.t2 is replicated');
+
+# Test owner of new table on subscriber matches the owner on publisher
+$node_publisher->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_subscriber->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_publisher->safe_psql('postgres', "SET SESSION AUTHORIZATION 'ddl_replication_user'; CREATE TABLE t5 (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
+is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+
+#TODO TEST certain DDLs are not replicated
+
+pass "DDL replication tests passed!";
+
+$node_subscriber->stop;
+$node_subscriber2->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.32.0
[application/octet-stream] 0002-Support-logical-logging-and-decoding-of-DDL-command-.patch (61.4K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/6-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch)
download | inline diff:
From 2816b9a8c3cae45915adb1e1bd3d595af85ae424 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:57:23 +0000
Subject: [PATCH 02/12] Support logical logging and decoding of DDL command
string.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current consumers.
The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on
the configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
Support logical decoding of the new WAL record xl_logical_ddl_message.
This is similar to the logical decoding of xl_logical_message. Tests
for this change are added in the test_decoding plugin.
---
contrib/test_decoding/Makefile | 4 +-
.../test_decoding/expected/ddlmessages.out | 42 ++++
contrib/test_decoding/sql/ddlmessages.sql | 28 +++
contrib/test_decoding/test_decoding.c | 61 +++++-
src/backend/access/rmgrdesc/Makefile | 1 +
.../access/rmgrdesc/logicalddlmsgdesc.c | 54 +++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 52 +++++
src/backend/commands/tablecmds.c | 43 +++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 99 +++++++++
src/backend/replication/logical/decode.c | 56 +++++
src/backend/replication/logical/logical.c | 91 ++++++++
.../replication/logical/reorderbuffer.c | 204 +++++++++++++++++-
src/backend/tcop/utility.c | 201 ++++++++++++++++-
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/include/access/rmgrlist.h | 1 +
src/include/commands/tablecmds.h | 3 +-
src/include/replication/ddlmessage.h | 47 ++++
src/include/replication/decode.h | 1 +
src/include/replication/output_plugin.h | 29 +++
src/include/replication/reorderbuffer.h | 39 ++++
src/test/regress/expected/publication.out | 16 +-
23 files changed, 1054 insertions(+), 21 deletions(-)
create mode 100644 contrib/test_decoding/expected/ddlmessages.out
create mode 100644 contrib/test_decoding/sql/ddlmessages.sql
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index b220906479..e58a76d3f1 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -5,7 +5,9 @@ PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- spill slot truncate stream stats twophase twophase_stream
+ spill slot truncate stream stats twophase twophase_stream \
+ ddlmessages
+
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
new file mode 100644
index 0000000000..79284f9def
--- /dev/null
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -0,0 +1,42 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+(6 rows)
+
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
new file mode 100644
index 0000000000..211497ee22
--- /dev/null
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -0,0 +1,28 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index 3736da6784..a44e1f79e3 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -76,6 +76,11 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
TransactionId xid,
const char *gid);
@@ -116,6 +121,11 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
int nrelations, Relation relations[],
@@ -141,6 +151,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
cb->message_cb = pg_decode_message;
+ cb->ddlmessage_cb = pg_decode_ddlmessage;
cb->filter_prepare_cb = pg_decode_filter_prepare;
cb->begin_prepare_cb = pg_decode_begin_prepare_txn;
cb->prepare_cb = pg_decode_prepare_txn;
@@ -153,6 +164,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pg_decode_stream_commit;
cb->stream_change_cb = pg_decode_stream_change;
cb->stream_message_cb = pg_decode_stream_message;
+ cb->stream_ddlmessage_cb = pg_decode_stream_ddlmessage;
cb->stream_truncate_cb = pg_decode_stream_truncate;
}
@@ -747,7 +759,8 @@ pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
appendStringInfo(ctx->out, "message: transactional: %d prefix: %s, sz: %zu content:",
@@ -756,6 +769,19 @@ pg_decode_message(LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+ appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_stream_start(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
@@ -936,7 +962,8 @@ pg_decode_stream_change(LogicalDecodingContext *ctx,
static void
pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
@@ -948,7 +975,35 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
else
{
appendStringInfo(ctx->out, "streaming message: transactional: %d prefix: %s, sz: %zu content:",
- transactional, prefix, sz);
+ transactional, prefix, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ }
+
+ OutputPluginWrite(ctx, true);
+}
+
+/*
+ * In streaming mode, we don't display the contents for transactional messages
+ * as the transaction can abort at a later point in time. We don't want users to
+ * see the message contents until the transaction is committed.
+ */
+static void
+pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char * role, const char * search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+
+ if (transactional)
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
+ transactional, prefix, role, search_path, sz);
+ }
+ else
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
}
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72fd86..b8e29e8df3 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000000..7a352d540a
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *role = xlrec->message + xlrec->prefix_size;
+ char *search_path = xlrec->message + xlrec->prefix_size + xlrec->role_size;
+ char *message = xlrec->message + xlrec->prefix_size + xlrec->role_size + xlrec->search_path_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
+ xlrec->transactional ? "transactional" : "non-transactional",
+ prefix, role, search_path, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed69244e3..6db9a593b8 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -29,6 +29,7 @@
#include "miscadmin.h"
#include "replication/decode.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "storage/standby.h"
#include "utils/builtins.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index b8ab1d8141..fad21a31d0 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1206,3 +1206,55 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
+
+/*
+ * Checks if DDL on relation (relid) need xlog for logical replication
+ */
+bool
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+{
+ List *allTablePubs = NIL;
+ List *tablePubs = NIL;
+ ListCell *lc;
+
+ /* Only replicate toplevel DDL command */
+ if (!isTopLevel)
+ return false;
+ if (relid == InvalidOid && !forAllTabPubOnly)
+ return false;
+
+ /*
+ * Log the DDL command if
+ * there is any FOR ALL TABLES publication with pubddl_database on
+ * or
+ * this TABLE belongs to any non FOR ALL publications with pubddl_table on
+ */
+ allTablePubs = GetAllTablesPublications();
+ foreach(lc, allTablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_database)
+ return true;
+ }
+
+ /*
+ * If forAllTabPubOnly is true (i.e. database level replication is required for the DDL
+ * to be logged), we can bail now since no publication has been found with pubddl_database on
+ */
+ if (forAllTabPubOnly)
+ return false;
+
+ tablePubs = GetRelationPublications(relid);
+ foreach(lc, tablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_table)
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..6d1487951f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -80,6 +80,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -1332,13 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(DropStmt *drop)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
+ bool ddlxlog = XLogLogicalInfoActive();
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1437,10 +1439,37 @@ RemoveRelations(DropStmt *drop)
/* Not there? */
if (!OidIsValid(relOid))
{
+ ddlxlog = false;
DropErrorMsgNonExistent(rel, relkind, drop->missing_ok);
continue;
}
+ /*
+ * Only log DROP RELATION cmd for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on or
+ * every relation to be dropped belongs to any non FOR ALL publications with pubddl_table on
+ */
+ if (ddlxlog)
+ {
+ Oid tableOid = InvalidOid;
+
+ if (relkind == RELKIND_RELATION)
+ tableOid = relOid;
+ else if (relkind == RELKIND_INDEX)
+ tableOid = IndexGetRelation(relOid, true);
+ /*
+ * Other relation types require database level ddl replication and are
+ * already logged in LogLogicalDDLCommand() if needed.
+ */
+ else
+ ddlxlog = false;
+
+ /* DROP RELATION or INDEX are allowed in table level DDL replication */
+ if (tableOid != InvalidOid &&
+ !ddl_need_xlog(tableOid, false, isTopLevel))
+ ddlxlog = false;
+ }
+
/*
* Decide if concurrent mode needs to be used here or not. The
* callback retrieved the rel's persistence for us.
@@ -1484,6 +1513,18 @@ RemoveRelations(DropStmt *drop)
add_exact_object_address(&obj, objects);
}
+ /* Log the Drop command for logical replication */
+ if (ddlxlog)
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
performMultipleDeletions(objects, drop->behavior, flags);
free_object_addresses(objects);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fdeb71..f3eeb67312 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000000..f93573079a
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
+ * Transactional messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ * Non-transactional messages are sent to the plugin at the time when the
+ * logical decoding reads them from XLOG. This also means that transactional
+ * messages won't be delivered if the transaction was rolled back but the
+ * non-transactional one will always be delivered.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
+{
+ xl_logical_ddl_message xlrec;
+ const char *role;
+
+ role = GetUserNameFromId(roleoid, false);
+
+ /*
+ * Force xid to be allocated if we're emitting a transactional message.
+ */
+ if (transactional)
+ {
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+ }
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.transactional = transactional;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.role_size = strlen(role) + 1;
+ xlrec.search_path_size = strlen(namespace_search_path) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, role), xlrec.role_size);
+ XLogRegisterData(namespace_search_path, xlrec.search_path_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427ba73..034c7f2413 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -603,6 +604,61 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
message->message + message->prefix_size);
}
+/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ Snapshot snapshot;
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (message->transactional &&
+ !SnapBuildProcessChange(builder, xid, buf->origptr))
+ return;
+ else if (!message->transactional &&
+ (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
+ SnapBuildXactNeedsSkip(builder, buf->origptr)))
+ return;
+
+ snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ message->transactional,
+ message->message,
+ /* first part of message is prefix */
+ message->message + message->prefix_size,
+ /* Second part of message is role*/
+ message->message + message->prefix_size + message->role_size,
+ /* Third part of message is search_path */
+ message->message_size,
+ message->message + message->prefix_size +
+ message->role_size + message->search_path_size);
+}
+
/*
* Consolidated commit record handling between the different form of commit
* records.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4273..3004f02433 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +94,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +226,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +243,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +261,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1220,6 +1231,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char *search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
@@ -1535,6 +1584,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char* search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f9089c..ca01336604 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,20 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.role != NULL)
+ pfree(change->data.ddlmsg.role);
+ change->data.ddlmsg.role = NULL;
+ if (change->data.ddlmsg.search_path != NULL)
+ pfree(change->data.ddlmsg.search_path);
+ change->data.ddlmsg.search_path = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -866,6 +880,64 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
}
+/*
+ * A transactional DDL message is queued to be processed upon commit and a
+ * non-transactional DDL message gets processed immediately.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
+{
+ if (transactional)
+ {
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ else
+ {
+ ReorderBufferTXN *txn = NULL;
+ volatile Snapshot snapshot_now = snapshot;
+
+ if (xid != InvalidTransactionId)
+ txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+
+ /* setup snapshot to allow catalog access */
+ SetupHistoricSnapshot(snapshot_now, NULL);
+ PG_TRY();
+ {
+ rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+
+ TeardownHistoricSnapshot(false);
+ }
+ PG_CATCH();
+ {
+ TeardownHistoricSnapshot(true);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+}
+
/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
@@ -1957,6 +2029,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message);
}
+/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
@@ -2335,6 +2430,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3708,6 +3807,53 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the role including the size */
+ memcpy(data, &role_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.role,
+ role_size);
+ data += role_size;
+
+ /* write the search_path including the size */
+ memcpy(data, &search_path_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.search_path,
+ search_path_size);
+ data += search_path_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4022,6 +4168,18 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4282,8 +4440,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4297,6 +4454,49 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+ Size role_size;
+ Size search_path_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read role */
+ memcpy(&role_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.role = MemoryContextAlloc(rb->context,
+ role_size);
+ memcpy(change->data.ddlmsg.role, data, role_size);
+ Assert(change->data.ddlmsg.role[role_size - 1] == '\0');
+ data += role_size;
+
+ /* read search_path */
+ memcpy(&search_path_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.search_path = MemoryContextAlloc(rb->context,
+ search_path_size);
+ memcpy(change->data.ddlmsg.search_path, data, search_path_size);
+ Assert(change->data.ddlmsg.search_path[search_path_size - 1] == '\0');
+ data += search_path_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcded55..5fe54f742a 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -62,6 +62,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
#include "storage/fd.h"
@@ -86,7 +87,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -987,7 +988,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel);
}
break;
@@ -1087,6 +1088,154 @@ standard_ProcessUtility(PlannedStmt *pstmt,
CommandCounterIncrement();
}
+/*
+ * Log a DDL command for logical replication
+ * Some DDLs are only replicated in Database Level DDL replication
+ * Some can be replicated in Table Level DDL replication.
+ *
+ * Currently we focus on supporting Database Level DDL replication
+ */
+static void
+LogLogicalDDLCommand(Node *parsetree, const char *queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* Fisrtly, commands that are only supported in Database level DDL replication */
+ case T_CreateSchemaStmt:
+ case T_CreateStmt:
+ case T_CreateForeignTableStmt:
+ case T_AlterDomainStmt:
+ case T_DefineStmt:
+ case T_CompositeTypeStmt:
+ case T_CreateEnumStmt:
+ case T_CreateRangeStmt:
+ case T_AlterEnumStmt:
+ case T_ViewStmt:
+ case T_CreateFunctionStmt:
+ case T_AlterFunctionStmt:
+ case T_CreateTrigStmt:
+ case T_CreateDomainStmt:
+ case T_CreateCastStmt:
+ case T_CreateOpClassStmt:
+ case T_CreateOpFamilyStmt:
+ case T_AlterOpFamilyStmt:
+ case T_AlterOperatorStmt:
+ case T_AlterTypeStmt:
+ case T_GrantStmt:
+ case T_AlterCollationStmt:
+ /*
+ * Log these stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+
+ /*
+ * Secondly, commands that may be allowed in Table level DDL replication.
+ * These are currently handled in the later execution path of the command.
+ * Because we need to get the relation id which readily available in later
+ * code path.
+ */
+ case T_AlterTableStmt:
+ case T_IndexStmt:
+ case T_RenameStmt: /* TODO */
+ case T_AlterOwnerStmt: /* TODO */
+ break;
+
+ /* DropStmt depends on the removeType */
+ case T_DropStmt:
+ {
+ DropStmt* stmt = (DropStmt*) parsetree;
+ switch (stmt->removeType)
+ {
+ /* Maybe allowed in Table level DDL replication, handled in later code path */
+ case OBJECT_INDEX:
+ case OBJECT_TABLE:
+ break;
+ /* Drop of sequence is by logical replication of sequences separately */
+ case OBJECT_SEQUENCE:
+ break;
+ /* Drop of other objects are allowed in Database level DDL replication only */
+ case OBJECT_VIEW:
+ case OBJECT_MATVIEW:
+ case OBJECT_FOREIGN_TABLE:
+ default:
+ /*
+ * Log these DropStmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+ }
+ }
+ /*
+ * Lastly, rule out DDLs we don't replicate yet in DDL replication
+ * Some of these can be supported, we just need to investigate and run tests.
+ */
+ case T_CreateExtensionStmt:
+ case T_AlterExtensionStmt:
+ case T_AlterExtensionContentsStmt:
+ case T_CreateFdwStmt:
+ case T_AlterFdwStmt:
+ case T_CreateForeignServerStmt:
+ case T_AlterForeignServerStmt:
+ case T_CreateUserMappingStmt:
+ case T_AlterUserMappingStmt:
+ case T_DropUserMappingStmt:
+ case T_ImportForeignSchemaStmt:
+ case T_RuleStmt:
+ case T_CreateSeqStmt:
+ case T_AlterSeqStmt:
+ case T_CreateTableAsStmt:
+ case T_RefreshMatViewStmt:
+ case T_CreatePLangStmt:
+ case T_CreateConversionStmt:
+ case T_CreateTransformStmt:
+ case T_AlterTSDictionaryStmt:
+ case T_AlterTSConfigurationStmt:
+ case T_AlterTableMoveAllStmt:
+ case T_AlterObjectDependsStmt:
+ case T_AlterObjectSchemaStmt:
+ case T_CommentStmt:
+ case T_DropOwnedStmt:
+ case T_AlterDefaultPrivilegesStmt:
+ case T_CreatePolicyStmt:
+ case T_AlterPolicyStmt:
+ case T_SecLabelStmt:
+ case T_CreateAmStmt:
+ case T_CreatePublicationStmt:
+ case T_AlterPublicationStmt:
+ case T_CreateSubscriptionStmt:
+ case T_AlterSubscriptionStmt:
+ case T_DropSubscriptionStmt:
+ case T_CreateStatsStmt:
+ case T_AlterStatsStmt:
+ break;
+ default:
+ break;
+ }
+}
+
/*
* The "Slow" variant of ProcessUtility should only receive statements
* supported by the event triggers facility. Therefore, we always
@@ -1119,6 +1268,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
switch (nodeTag(parsetree))
{
/*
@@ -1321,6 +1477,23 @@ ProcessUtilitySlow(ParseState *pstate,
EventTriggerAlterTableStart(parsetree);
EventTriggerAlterTableRelid(relid);
+ /*
+ * Log the ALTER TABLE command if
+ * There is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
/* ... and do it */
AlterTable(atstmt, lockmode, &atcontext);
@@ -1539,6 +1712,24 @@ ProcessUtilitySlow(ParseState *pstate,
/* ... and do it */
EventTriggerAlterTableStart(parsetree);
+
+ /*
+ * Log CREATE INDEX cmd for logical replication if
+ * there is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on.
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
address =
DefineIndex(relid, /* OID of heap relation */
stmt,
@@ -1761,7 +1952,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt((DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -1982,7 +2173,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
{
switch (stmt->removeType)
{
@@ -1997,7 +2188,7 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(stmt);
+ RemoveRelations(pstate, stmt, isTopLevel);
break;
default:
RemoveObjects(stmt);
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17bb4c..792f438959 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721c97..9de3b8f2eb 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 5d4037f26e..68781365de 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -18,6 +18,7 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "storage/lock.h"
#include "utils/relcache.h"
@@ -27,7 +28,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(DropStmt *drop);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000000..1e8ef22296
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ bool transactional; /* is message transactional? */
+ Size prefix_size; /* length of prefix */
+ Size role_size; /* length of the role that executes the DDL command */
+ Size search_path_size; /* length of the search path */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ * and null-terminated role of length role_size
+ * and null-terminated search_path of length search_path_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
+ size_t size, bool transactional);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65cf7..427a7b997d 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e697..5b1c245b72 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -88,6 +88,19 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
Size message_size,
const char *message);
+/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Filter changes by origin.
*/
@@ -199,6 +212,20 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
Size message_size,
const char *message);
+/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Callback for streaming truncates from in-progress transactions.
*/
@@ -219,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f877e5..dd89e08efc 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ char *role;
+ char *search_path;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +441,17 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +518,18 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size message_size, const char *message);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index e9cea11a5b..5b4bee74cb 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | t | f | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
--
2.32.0
[application/octet-stream] 0006-Add-couple-tests-for-DDL-replication-case.patch (4.5K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/7-0006-Add-couple-tests-for-DDL-replication-case.patch)
download | inline diff:
From ac926d5c2888577b44c935f5a4c136b1a5d29f19 Mon Sep 17 00:00:00 2001
From: Borui Yang <boruiyan@amazon.com>
Date: Mon, 11 Apr 2022 18:02:21 +0000
Subject: [PATCH 06/12] Add couple tests for DDL replication case
---
src/test/subscription/t/030_rep_ddls.pl | 80 +++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 3b15c6d9f0..4e05212068 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -268,6 +268,86 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+# TEST Create DomainStmt
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(t), 'CreateDomain Stmt is replicatted');
+
+# TEST AlterDomainStmt
+$node_publisher->safe_psql('postgres', "Alter domain s1.space_check drop not null;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(f), 'ALTER DOMAIN Stmt is replicated');
+
+#TEST DEFINE Stmt
+$node_publisher->safe_psql('postgres', "CREATE AGGREGATE s1.inc_sum(int) (sfunc = int4pl,stype = int,initcond = 10);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='inc_sum';");
+is($result, qq(1), 'Define stmt is replicated');
+
+#TEST CompositeTypeStmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.compfoo AS (f1 int, f2 text);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='compfoo';");
+is($result, qq(1), 'CompositeType Stmt is replicated');
+
+#TEST CreateEnum Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.mood AS ENUM ('sad', 'ok', 'happy');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='mood';");
+is($result, qq(1), 'CreateEnumType Stmt is replicated');
+
+#TEST AlterEnum Stmt
+$node_publisher->safe_psql('postgres', "ALTER TYPE s1.mood RENAME VALUE 'sad' to 'fine';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_enum e, pg_catalog.pg_type t WHERE e.enumtypid = t.oid AND t.typname='mood' AND e.enumlabel='fine';");
+is($result, qq(1), 'AlterEnumType Stmt is replicated');
+
+#TEST CreateRange Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE floatrange AS RANGE (subtype = float8,subtype_diff = float8mi);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='floatrange';");
+is($result, qq(1), 'CreateRange Stmt is replicated');
+
+#TEST VIEW Stmt
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.vista AS SELECT 'Hello World';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_class c where c.relname='vista';");
+is($result, qq(1), 'VIEW Stmt is replicated');
+
+#TEST CreateFunction Stmt
+$node_publisher->safe_psql('postgres', "CREATE FUNCTION s1.add(a integer, b integer) RETURNS integer LANGUAGE SQL IMMUTABLE RETURN a + b;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='add';");
+is($result, qq(1), 'CreateFunction Stmt is replicated');
+
+#TEST CreateCast Stmt
+$node_publisher->safe_psql('postgres', "CREATE CAST (int AS int4) WITH FUNCTION add(int,int) AS ASSIGNMENT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
+is($result, qq(1), 'CreateCast Stmt is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch (4.8K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/8-0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch)
download | inline diff:
From 4bbb0676705523a724e3a86436906544e6ac5e06 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 29 Apr 2022 16:57:03 +0000
Subject: [PATCH 08/12] Fail replication worker on DDL command that rewrites
table using volatile functions, such as ALTER TABLE tab ADD COLUMN col
DEFAULT volatile_expr. This is to avoid data mismatch compared to the
publisher. We can potentially unblock this type of command when table rewrite
is supported in logical replication.
---
src/backend/replication/logical/worker.c | 44 ++++++++++++++++++++++++
src/test/subscription/t/030_rep_ddls.pl | 22 ++++++++++++
2 files changed, 66 insertions(+)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 905f47faf5..01db102e3c 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -157,6 +157,7 @@
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
+#include "parser/parse_expr.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -2547,6 +2548,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
commandTag = CreateCommandTag((Node *)command);
+ /* The following DDL commands need special handling */
+
/*
* Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
@@ -2594,6 +2597,47 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
sstmt->intoClause->skipData = true;
}
}
+ /*
+ * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
+ * Until we support logical replication of table rewrite, see ATRewriteTables()
+ * for details on table rewrite.
+ */
+ else if (commandTag == CMDTAG_ALTER_TABLE)
+ {
+ AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
+ ListCell *lc;
+
+ foreach(lc, atstmt->cmds)
+ {
+ AlterTableCmd *cmd = lfirst_node(AlterTableCmd, lc);
+
+ if (cmd->subtype == AT_AddColumn)
+ {
+ ColumnDef *colDef;
+ ListCell *c;
+
+ colDef = castNode(ColumnDef, cmd->def);
+ foreach(c, colDef->constraints)
+ {
+ Constraint *con = lfirst_node(Constraint, c);
+
+ if (con->contype == CONSTR_DEFAULT)
+ {
+ Node *expr;
+ ParseState *pstate = make_parsestate(NULL);
+
+ expr = transformExpr(pstate, copyObject(con->raw_expr), EXPR_KIND_COLUMN_DEFAULT);
+ if (contain_volatile_functions(expr))
+ {
+ elog(ERROR,
+ "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
+ cmdstr);
+ }
+ }
+ }
+ }
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 34b9d51eb1..b4df1bfefd 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -386,7 +386,29 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
is($result, qq(1), 'DDLs in procedure are replicated');
+# Test Alter table alter column type stmt is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN name type TEXT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT data_type FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'name';");
+is($result, qq(text), 'Alter table column type stmt is replicated');
+
+# Test Alter table add column default 0.01 is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN non_volatile double precision DEFAULT 0.01;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
+is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+
#TODO TEST certain DDLs are not replicated
+# Test DDL statement that rewrites table with volatile functions are not replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'volatile';");
+is($result, qq(1), 'Alter table add column default random() is executed on the publisher DB.');
+
+$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch (8.3K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/9-0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch)
download | inline diff:
From 7e66412f92674dcb1adf29140aac692340491e00 Mon Sep 17 00:00:00 2001
From: PonyboyYbr <94borelyang@gmail.com>
Date: Wed, 11 May 2022 15:46:40 -0700
Subject: [PATCH 09/12] Support replication of DDL type T_RenameStmt: table
rename is allowed in both database level and table level DDL replication.
Rename of other objects are only allowed in database level DDL replication.
Co-authored-by: Borui Yang <boruiyan@amazon.com>
---
src/backend/commands/alter.c | 5 ++--
src/backend/commands/tablecmds.c | 15 ++++++++++-
src/backend/tcop/utility.c | 20 ++++++++++++---
src/include/commands/alter.h | 3 ++-
src/include/commands/tablecmds.h | 2 +-
src/test/subscription/t/030_rep_ddls.pl | 33 +++++++++++++++++++++++--
6 files changed, 67 insertions(+), 11 deletions(-)
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 5456b8222b..1ca8f99e0c 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -328,7 +328,8 @@ AlterObjectRename_internal(Relation rel, Oid objectId, const char *new_name)
* Return value is the address of the renamed object.
*/
ObjectAddress
-ExecRenameStmt(RenameStmt *stmt)
+ExecRenameStmt(ParseState *pstate, RenameStmt *stmt,
+ bool isCompleteQuery)
{
switch (stmt->renameType)
{
@@ -354,7 +355,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_MATVIEW:
case OBJECT_INDEX:
case OBJECT_FOREIGN_TABLE:
- return RenameRelation(stmt);
+ return RenameRelation(pstate, stmt, isCompleteQuery);
case OBJECT_COLUMN:
case OBJECT_ATTRIBUTE:
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3c9b6409ca..65386f2641 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3820,11 +3820,12 @@ RenameConstraint(RenameStmt *stmt)
* RENAME
*/
ObjectAddress
-RenameRelation(RenameStmt *stmt)
+RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
{
bool is_index_stmt = stmt->renameType == OBJECT_INDEX;
Oid relid;
ObjectAddress address;
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/*
* Grab an exclusive lock on the target table, index, sequence, view,
@@ -3872,6 +3873,18 @@ RenameRelation(RenameStmt *stmt)
is_index_stmt = obj_is_index;
}
+ if (ddlxlog &&
+ ddl_need_xlog(relid, false))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
/* Do the work */
RenameRelationInternal(relid, stmt->newname, false, is_index_stmt);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index e9e7567209..1abd77a60e 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1002,7 +1002,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecRenameStmt(stmt);
+ ExecRenameStmt(pstate, stmt, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1195,8 +1195,20 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
case T_AlterTableStmt:
case T_IndexStmt:
- case T_RenameStmt: /* TODO */
- case T_AlterOwnerStmt: /* TODO */
+ case T_RenameStmt:
+ {
+ RenameStmt *stmt = (RenameStmt *) parsetree;
+ if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ }
+ case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
break;
/* DropStmt depends on the removeType */
@@ -2008,7 +2020,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_RenameStmt:
- address = ExecRenameStmt((RenameStmt *) parsetree);
+ address = ExecRenameStmt(pstate, (RenameStmt *) parsetree, isCompleteQuery);
break;
case T_AlterObjectDependsStmt:
diff --git a/src/include/commands/alter.h b/src/include/commands/alter.h
index 52f5e6f6d2..df9333eb93 100644
--- a/src/include/commands/alter.h
+++ b/src/include/commands/alter.h
@@ -17,9 +17,10 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "utils/relcache.h"
-extern ObjectAddress ExecRenameStmt(RenameStmt *stmt);
+extern ObjectAddress ExecRenameStmt(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern ObjectAddress ExecAlterObjectDependsStmt(AlterObjectDependsStmt *stmt,
ObjectAddress *refAddress);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 24106de2e5..f96bb57d56 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -74,7 +74,7 @@ extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
-extern ObjectAddress RenameRelation(RenameStmt *stmt);
+extern ObjectAddress RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern void RenameRelationInternal(Oid myrelid,
const char *newrelname, bool is_internal,
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index b4df1bfefd..51126b489b 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -30,7 +30,6 @@ $node_subscriber->safe_psql('postgres', $ddl);
$node_subscriber2->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-
# mypub has pubddl_database on
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION mypub FOR ALL TABLES;");
@@ -269,7 +268,7 @@ $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;")
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
# TEST Create DomainStmt
-$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\\s');");
$node_publisher->wait_for_catchup('mysub');
@@ -348,6 +347,36 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST RenameStmt for FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER FUNCTION add RENAME TO plus;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='plus';");
+is($result, qq(1), 'RENAME FUNCTION Stmt is replicated');
+
+#TEST RenameStmt for table
+$node_publisher->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_publisher->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1',
+ "CREATE PUBLICATION mypub3 FOR TABLE t7 with (ddl = 'table');");
+my $publisher_connstr_db1 = $node_publisher->connstr . ' dbname=db1';
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1',
+ "CREATE SUBSCRIPTION mysub3 CONNECTION '$publisher_connstr_db1' PUBLICATION mypub3;"
+);
+$node_publisher->wait_for_catchup('mysub3');
+$node_publisher->safe_psql('db1', "ALTER TABLE t7 RENAME TO newt7;");
+$node_publisher->safe_psql('db1', "ALTER TABLE t8 RENAME TO newt8;");
+$node_publisher->wait_for_catchup('mysub3');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt7';");
+is($result, qq(1), 'Rename t7 to newt7 is replicated');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt8';");
+is($result, qq(0), 'Rename t8 to newt8 is not replicated');
+
#TEST DDL in function
$node_publisher->safe_psql('postgres', qq{
CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
--
2.32.0
[application/octet-stream] 0010-Handle-partitioned-table-creation-on-the-apply-worke.patch (11.5K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/10-0010-Handle-partitioned-table-creation-on-the-apply-worke.patch)
download | inline diff:
From 59c096935de29c1df14c80fc25fd58195886bfd7 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 13 May 2022 19:47:44 +0000
Subject: [PATCH 10/12] Handle partitioned table creation on the apply worker:
whether a partitioned table should be added to pg_subscription_rel catalog
depends on the setting of publish_via_partition_root of the publication. Thus
we need to connect to the source DB and check whehter the partitioned table
should be subscribed.
---
src/backend/commands/subscriptioncmds.c | 54 +++++++++++++++++++++-
src/backend/replication/logical/worker.c | 35 ++++++++++++---
src/include/commands/subscriptioncmds.h | 3 ++
src/test/regress/expected/psql.out | 6 +--
src/test/subscription/t/030_rep_ddls.pl | 57 ++++++++++++++++++++++++
5 files changed, 145 insertions(+), 10 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 83e6eae855..25d0a23db9 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -23,7 +23,6 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/objectaddress.h"
-#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
@@ -746,6 +745,59 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
return myself;
}
+/*
+ * Check if a partitioned table is being published by any publication subscirbed by the subscription.
+ * Whether a partitioned table is published also depends on the publication option
+ * publish_via_partition_root. But the subscriber doesn't know the setting of publish_via_partition_root,
+ * this is why we need to check the source DB so that we can decide whether to subscribe to the partitioned
+ * table (could be either root or leaf table) during replication of create partitioned table.
+ */
+bool
+IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name)
+{
+ char *err;
+ WalRcvExecResult *res;
+ StringInfoData cmd;
+ Oid tableRow[1] = {BOOLOID};
+ bool result;
+
+ WalReceiverConn *wrconn;
+ /* Load the library providing us libpq calls. */
+ load_file("libpqwalreceiver", false);
+
+ /* Try to connect to the publisher. */
+ wrconn = walrcv_connect(sub->conninfo, true, sub->name, &err);
+ if (!wrconn)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not connect to the publisher: %s", err)));
+
+ PG_TRY();
+ {
+ initStringInfo(&cmd);
+ appendStringInfoString(&cmd, "SELECT TRUE\n"
+ " FROM pg_catalog.pg_publication_tables t\n"
+ " WHERE t.pubname IN (");
+ get_publications_str(sub->publications, &cmd, true);
+ appendStringInfoString(&cmd, ") AND t.schemaname = '");
+ appendStringInfoString(&cmd, schema_name);
+ appendStringInfoString(&cmd, "' AND t.tablename = '");
+ appendStringInfoString(&cmd, table_name);
+ appendStringInfoString(&cmd, "'");
+
+ res = walrcv_exec(wrconn, cmd.data, 1, tableRow);
+ pfree(cmd.data);
+ result = tuplestore_tuple_count(res->tuplestore) > 0;
+ }
+ PG_FINALLY();
+ {
+ walrcv_disconnect(wrconn);
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
static void
AlterSubscription_refresh(Subscription *sub, bool copy_data,
List *validate_publications)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 01db102e3c..610147ca77 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -143,6 +143,7 @@
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_tablespace.h"
+#include "commands/subscriptioncmds.h"
#include "commands/tablecmds.h"
#include "commands/tablespace.h"
#include "commands/trigger.h"
@@ -2545,6 +2546,7 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
bool snapshot_set = false;
char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ bool is_partitioned_table = false;
commandTag = CreateCommandTag((Node *)command);
@@ -2561,6 +2563,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
+ if (cstmt->inhRelations != NIL || cstmt->partspec != NULL)
+ is_partitioned_table = true;
}
else if (commandTag == CMDTAG_CREATE_TABLE_AS)
{
@@ -2762,12 +2766,31 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
if (relid != InvalidOid)
{
- AddSubscriptionRelState(MySubscription->oid, relid,
- SUBREL_STATE_INIT,
- InvalidXLogRecPtr);
- ereport(DEBUG1,
- (errmsg_internal("table \"%s\" added to subscription \"%s\"",
- relname, MySubscription->name)));
+ bool subscribe_table = true;
+
+ if (is_partitioned_table)
+ {
+ Relation rel = RelationIdGetRelation(relid);
+ char *table_name = RelationGetRelationName(rel);
+ char *schema_name = get_namespace_name(RelationGetNamespace(rel));
+ /*
+ * Connect to the source DB and check whehter the partitioned table should be subscribed.
+ * Because it depends on the setting of publish_via_partition_root, which the subscription
+ * doesn't know.
+ */
+ subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
+ RelationClose(rel);
+ }
+
+ if (subscribe_table)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
}
}
}
diff --git a/src/include/commands/subscriptioncmds.h b/src/include/commands/subscriptioncmds.h
index 2cbe7d7b65..451953af60 100644
--- a/src/include/commands/subscriptioncmds.h
+++ b/src/include/commands/subscriptioncmds.h
@@ -16,6 +16,7 @@
#define SUBSCRIPTIONCMDS_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_subscription.h"
#include "parser/parse_node.h"
extern ObjectAddress CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
@@ -26,4 +27,6 @@ extern void DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel);
extern ObjectAddress AlterSubscriptionOwner(const char *name, Oid newOwnerId);
extern void AlterSubscriptionOwner_oid(Oid subid, Oid newOwnerId);
+extern bool IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name);
+
#endif /* SUBSCRIPTIONCMDS_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1241..3429e34339 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 51126b489b..a03d598266 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -431,6 +431,63 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+# Test partitioned table creation is replicated based on the setting of publish_via_partition_root
+$node_publisher->safe_psql('postgres', qq(
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table is created and populated');
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table and data is replicated');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(0), 'Root table of the partitioned table is not subscribed since publish_via_partition_root is false by default');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(2), 'Only leaf tables of the partitioned table are subscribed since publish_via_partition_root is false by default');
+
+# Test only root partition table is subscribed when publish_via_partition_root is enabled
+$node_publisher->safe_psql('postgres', qq(
+ DROP TABLE s1.test_part_a;
+ ALTER PUBLICATION mypub SET (publish_via_partition_root = true);
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(1), 'Only root table of the partitioned table is subscribed since publish_via_partition_root is enabled');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(0), 'Leaf tables of the partitioned table are not subscribed since publish_via_partition_root is enabled');
+
#TODO TEST certain DDLs are not replicated
# Test DDL statement that rewrites table with volatile functions are not replicated
$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
--
2.32.0
[application/octet-stream] 0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch (17.5K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/11-0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch)
download | inline diff:
From 13f1992172fb5bad19bfae83aee3df7c8d3cfa5d Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 15 Apr 2022 02:02:10 +0000
Subject: [PATCH 07/12] Enable logging and replication of DDLs executed inside
function and procedures. Also fixed a bug where DDL with SUBCOMMAND gets
logged twice.
---
.../test_decoding/expected/ddlmessages.out | 47 +++++++++++++++++--
contrib/test_decoding/sql/ddlmessages.sql | 19 +++++++-
src/backend/catalog/pg_publication.c | 5 +-
src/backend/commands/tablecmds.c | 6 +--
src/backend/tcop/utility.c | 26 +++++-----
src/include/catalog/pg_publication.h | 2 +-
src/include/commands/tablecmds.h | 3 +-
src/test/subscription/t/030_rep_ddls.pl | 38 +++++++++++++++
8 files changed, 121 insertions(+), 25 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 0376f36c24..823029d03d 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -11,7 +11,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d
init
(1 row)
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -24,9 +24,9 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
@@ -34,6 +34,44 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+SELECT func_ddl ('tab_from_func');
+ func_ddl
+----------
+
+(1 row)
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
+ LANGUAGE plpgsql;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ BEGIN
+ table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
+ table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
+ COMMIT
+(7 rows)
+
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
@@ -41,4 +79,5 @@ SELECT pg_drop_replication_slot('regression_slot');
(1 row)
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index c23610f9b4..3082671488 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -9,7 +9,7 @@ SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -25,7 +25,24 @@ ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+
+SELECT func_ddl ('tab_from_func');
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index fad21a31d0..d5e4f3713d 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1211,15 +1211,12 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
* Checks if DDL on relation (relid) need xlog for logical replication
*/
bool
-ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly)
{
List *allTablePubs = NIL;
List *tablePubs = NIL;
ListCell *lc;
- /* Only replicate toplevel DDL command */
- if (!isTopLevel)
- return false;
if (relid == InvalidOid && !forAllTabPubOnly)
return false;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6d1487951f..3c9b6409ca 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1333,14 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
- bool ddlxlog = XLogLogicalInfoActive();
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1466,7 +1466,7 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
/* DROP RELATION or INDEX are allowed in table level DDL replication */
if (tableOid != InvalidOid &&
- !ddl_need_xlog(tableOid, false, isTopLevel))
+ !ddl_need_xlog(tableOid, false))
ddlxlog = false;
}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 29381b53b6..e9e7567209 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -87,7 +87,8 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -988,7 +989,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(pstate, stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1128,7 +1129,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1170,7 +1171,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1221,7 +1222,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1318,7 +1319,7 @@ ProcessUtilitySlow(ParseState *pstate,
* Consider logging the DDL command if logical logging is enabled and this is
* a complete top level query.
*/
- if (XLogLogicalInfoActive() && isTopLevel)
+ if (XLogLogicalInfoActive())
LogLogicalDDLCommand(parsetree, queryString);
}
@@ -1530,7 +1531,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1766,7 +1768,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on.
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1999,7 +2002,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel, isCompleteQuery);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -2220,7 +2223,8 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery)
{
switch (stmt->removeType)
{
@@ -2235,7 +2239,7 @@ ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(pstate, stmt, isTopLevel);
+ RemoveRelations(pstate, stmt, isCompleteQuery);
break;
default:
RemoveObjects(stmt);
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index a94e152256..8c114b2447 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -166,6 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
-extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 68781365de..24106de2e5 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -28,7 +28,8 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop,
+ bool isCompleteQuery);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 4e05212068..34b9d51eb1 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -348,6 +348,44 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST DDL in function
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+\$\$
+LANGUAGE plpgsql;});
+
+$node_publisher->safe_psql('postgres', "SELECT func_ddl('func_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.func_table where c3 = 22;");
+is($result, qq(1), 'DDLs in function are replicated');
+
+#TEST DDL in procedure
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE procedure proc_ddl (tname varchar(20))
+LANGUAGE plpgsql AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END \$\$;});
+
+$node_publisher->safe_psql('postgres', "CALL proc_ddl('proc_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
+is($result, qq(1), 'DDLs in procedure are replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0012-Change-the-logging-format-of-logical-ddlmessage-to-t.patch (36.9K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/12-0012-Change-the-logging-format-of-logical-ddlmessage-to-t.patch)
download | inline diff:
From ce7e6a0087e626d2279fe0ca3472d7eea45fa851 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Sun, 12 Jun 2022 05:28:02 +0000
Subject: [PATCH 12/12] Change the logging format of logical ddlmessage to that
of the serialization of the parsetree of the DDL command using
nodeToString(), the apply worker deserializes the ddlmessage by calling
stringToNode(). Detailed changes: - Adding serialization/deserialization
functions in outfuncs.c/readfuncs.c for CreateTableStmt, AlterTableStmt,
DropStmt, CreateFunctionStmt and AlterFunctionStmt. - Modified the
serialization process to always schema qualify object names, this is done
by outQualifiedName() and a change in _outRangeVar(). - Change the input of
LogLogicalDDLMessage() to use nodeToString(parsetree). - Change the apply
worker to call stringToNode(ddlmessage) to deserialize the parsetree and
then directly execute the parsetree.
Purpose of the commit is to explore the new logging format, lot of the
existing tests in 030_rep_ddls.pl fails due to missing serialization/
deserialization support of their DDL parsetree types. Only CreateTableStmt,
AlterTableStmt, DropStmt, CreateFunctionStmt are tested to be working
at the moment.
---
src/backend/commands/tablecmds.c | 10 +-
src/backend/nodes/outfuncs.c | 181 ++++++++-
src/backend/nodes/readfuncs.c | 452 +++++++++++++++++++++++
src/backend/replication/logical/worker.c | 345 ++++++++---------
src/backend/tcop/utility.c | 49 ++-
5 files changed, 846 insertions(+), 191 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 94e350b80d..87978b7523 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1517,10 +1517,11 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
if (ddlxlog)
{
const char* prefix = "";
+ char* parsetree_str = nodeToString((Node *) drop);
LogLogicalDDLMessage(prefix,
GetUserId(),
- pstate->p_sourcetext,
- strlen(pstate->p_sourcetext));
+ parsetree_str,
+ strlen(parsetree_str));
}
performMultipleDeletions(objects, drop->behavior, flags);
@@ -3875,10 +3876,11 @@ RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
ddl_need_xlog(relid, false))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString((Node *) stmt);
LogLogicalDDLMessage(prefix,
GetUserId(),
- pstate->p_sourcetext,
- strlen(pstate->p_sourcetext));
+ parsetree_str,
+ strlen(parsetree_str));
}
/* Do the work */
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index ce12915592..d302c63923 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -29,12 +29,14 @@
#include <ctype.h>
+#include "catalog/namespace.h"
#include "lib/stringinfo.h"
#include "miscadmin.h"
#include "nodes/extensible.h"
#include "nodes/pathnodes.h"
#include "nodes/plannodes.h"
#include "utils/datum.h"
+#include "utils/lsyscache.h"
#include "utils/rel.h"
static void outChar(StringInfo str, char c);
@@ -210,6 +212,34 @@ outChar(StringInfo str, char c)
outToken(str, in);
}
+/*
+ * Convert a possibly qualified name (list of String nodes)
+ * We'll qualify the name if the input is not qualified.
+ */
+static void
+outQualifiedName(StringInfo str, List *names)
+{
+ appendStringInfoChar(str, '(');
+ if (list_length(names) > 0)
+ {
+ Oid namespaceId;
+ char *schemaname;
+ char *objname;
+
+ namespaceId = QualifiedNameGetCreationNamespace(names, &objname);
+ schemaname = get_namespace_name(namespaceId);
+
+ appendStringInfoChar(str, '"');
+ appendStringInfoString(str, schemaname);
+ appendStringInfoChar(str, '"');
+ appendStringInfoChar(str, ' ');
+ appendStringInfoChar(str, '"');
+ appendStringInfoString(str, objname);
+ appendStringInfoChar(str, '"');
+ }
+ appendStringInfoChar(str, ')');
+}
+
static void
_outList(StringInfo str, const List *node)
{
@@ -1090,7 +1120,16 @@ _outRangeVar(StringInfo str, const RangeVar *node)
* we deliberately ignore catalogname here, since it is presently not
* semantically meaningful
*/
- WRITE_STRING_FIELD(schemaname);
+ if (node->schemaname == NULL)
+ {
+ Oid schema_oid = RangeVarGetCreationNamespace(node);
+ char *schema_name = get_namespace_name(schema_oid);
+
+ appendStringInfoString(str, " :schemaname ");
+ appendStringInfoString(str, schema_name);
+ }
+ else
+ WRITE_STRING_FIELD(schemaname);
WRITE_STRING_FIELD(relname);
WRITE_BOOL_FIELD(inh);
WRITE_CHAR_FIELD(relpersistence);
@@ -2888,6 +2927,18 @@ _outCreateStmt(StringInfo str, const CreateStmt *node)
_outCreateStmtInfo(str, (const CreateStmt *) node);
}
+static void
+_outCreateTableAsStmt(StringInfo str, const CreateTableAsStmt *node)
+{
+ WRITE_NODE_TYPE("CREATETABLEASSTMT");
+
+ WRITE_NODE_FIELD(query);
+ WRITE_NODE_FIELD(into);
+ WRITE_ENUM_FIELD(objtype, ObjectType);
+ WRITE_BOOL_FIELD(is_select_into);
+ WRITE_BOOL_FIELD(if_not_exists);
+}
+
static void
_outCreateForeignTableStmt(StringInfo str, const CreateForeignTableStmt *node)
{
@@ -2899,6 +2950,106 @@ _outCreateForeignTableStmt(StringInfo str, const CreateForeignTableStmt *node)
WRITE_NODE_FIELD(options);
}
+static void
+_outAlterTableStmt(StringInfo str, const AlterTableStmt *node)
+{
+ WRITE_NODE_TYPE("ALTERTABLESTMT");
+
+ WRITE_NODE_FIELD(relation);
+ WRITE_NODE_FIELD(cmds);
+ WRITE_ENUM_FIELD(objtype, ObjectType);
+ WRITE_BOOL_FIELD(missing_ok);
+}
+
+static void
+_outAlterTableCmd(StringInfo str, const AlterTableCmd *node)
+{
+ WRITE_NODE_TYPE("ALTERTABLECMD");
+
+ WRITE_ENUM_FIELD(subtype, AlterTableType);
+ WRITE_STRING_FIELD(name);
+ WRITE_INT_FIELD(num);
+ WRITE_NODE_FIELD(newowner);
+ WRITE_NODE_FIELD(def);
+ WRITE_ENUM_FIELD(behavior, DropBehavior);
+ WRITE_BOOL_FIELD(missing_ok);
+}
+
+static void
+_outDropStmt(StringInfo str, const DropStmt *node)
+{
+ WRITE_NODE_TYPE("DROPSTMT");
+
+ //WRITE_NODE_FIELD(objects);
+ appendStringInfoString(str, " :objects ");
+ /* Qualify and OUT object names */
+ if (node->objects)
+ {
+ ListCell *lc;
+
+ appendStringInfoChar(str, '(');
+ foreach(lc, node->objects)
+ {
+ List *names = (List *) lfirst(lc);
+ outQualifiedName(str, names);
+ }
+ appendStringInfoChar(str, ')');
+ }
+ else
+ appendStringInfoString(str, "<>");
+
+ WRITE_ENUM_FIELD(removeType, ObjectType);
+ WRITE_ENUM_FIELD(behavior, DropBehavior);
+ WRITE_BOOL_FIELD(missing_ok);
+ WRITE_BOOL_FIELD(concurrent);
+}
+
+static void
+_outCreateFunctionStmt(StringInfo str, const CreateFunctionStmt *node)
+{
+ WRITE_NODE_TYPE("CREATEFUNCTIONSTMT");
+
+ WRITE_BOOL_FIELD(is_procedure);
+ WRITE_BOOL_FIELD(replace);
+ appendStringInfoString(str, " :funcname ");
+ outQualifiedName(str, node->funcname);
+ WRITE_NODE_FIELD(parameters);
+ WRITE_NODE_FIELD(returnType);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(sql_body);
+}
+
+static void
+_outFunctionParameter(StringInfo str, const FunctionParameter *node)
+{
+ WRITE_NODE_TYPE("FUNCTIONPARAMETER");
+
+ WRITE_STRING_FIELD(name);
+ WRITE_NODE_FIELD(argType);
+ WRITE_ENUM_FIELD(mode, FunctionParameterMode);
+ WRITE_NODE_FIELD(defexpr);
+}
+
+static void
+_outAlterFunctionStmt(StringInfo str, const AlterFunctionStmt *node)
+{
+ WRITE_NODE_TYPE("ALTERFUNCTIONSTMT");
+
+ WRITE_ENUM_FIELD(objtype, ObjectType);
+ WRITE_NODE_FIELD(func);
+ WRITE_NODE_FIELD(actions);
+}
+
+static void
+_outRoleSpec(StringInfo str, const RoleSpec *node)
+{
+ WRITE_NODE_TYPE("ROLESPEC");
+
+ WRITE_ENUM_FIELD(type, RoleSpecType);
+ WRITE_STRING_FIELD(rolename);
+ WRITE_INT_FIELD(location);
+}
+
static void
_outImportForeignSchemaStmt(StringInfo str, const ImportForeignSchemaStmt *node)
{
@@ -3134,7 +3285,9 @@ _outTypeName(StringInfo str, const TypeName *node)
{
WRITE_NODE_TYPE("TYPENAME");
- WRITE_NODE_FIELD(names);
+ //WRITE_NODE_FIELD(names);
+ appendStringInfoString(str, " :names ");
+ outQualifiedName(str, node->names);
WRITE_OID_FIELD(typeOid);
WRITE_BOOL_FIELD(setof);
WRITE_BOOL_FIELD(pct_type);
@@ -4543,9 +4696,33 @@ outNode(StringInfo str, const void *obj)
case T_CreateStmt:
_outCreateStmt(str, obj);
break;
+ case T_CreateTableAsStmt:
+ _outCreateTableAsStmt(str, obj);
+ break;
case T_CreateForeignTableStmt:
_outCreateForeignTableStmt(str, obj);
break;
+ case T_AlterTableStmt:
+ _outAlterTableStmt(str, obj);
+ break;
+ case T_AlterTableCmd:
+ _outAlterTableCmd(str, obj);
+ break;
+ case T_DropStmt:
+ _outDropStmt(str, obj);
+ break;
+ case T_CreateFunctionStmt:
+ _outCreateFunctionStmt(str, obj);
+ break;
+ case T_FunctionParameter:
+ _outFunctionParameter(str, obj);
+ break;
+ case T_AlterFunctionStmt:
+ _outAlterFunctionStmt(str, obj);
+ break;
+ case T_RoleSpec:
+ _outRoleSpec(str, obj);
+ break;
case T_ImportForeignSchemaStmt:
_outImportForeignSchemaStmt(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 6a05b69415..71556d33a8 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2938,6 +2938,426 @@ _readPartitionRangeDatum(void)
READ_DONE();
}
+/*
+ * _readCreateStmt
+ */
+static CreateStmt*
+_readCreateStmt(void)
+{
+ READ_LOCALS(CreateStmt);
+
+ READ_NODE_FIELD(relation);
+ READ_NODE_FIELD(tableElts);
+ READ_NODE_FIELD(inhRelations);
+ READ_NODE_FIELD(partspec);
+ READ_NODE_FIELD(partbound);
+ READ_NODE_FIELD(ofTypename);
+ READ_NODE_FIELD(constraints);
+ READ_NODE_FIELD(options);
+ READ_ENUM_FIELD(oncommit, OnCommitAction);
+ READ_STRING_FIELD(tablespacename);
+ READ_STRING_FIELD(accessMethod);
+ READ_BOOL_FIELD(if_not_exists);
+
+ READ_DONE();
+}
+
+/*
+ * _readCreateTableAsStmt
+ */
+static CreateTableAsStmt*
+_readCreateTableAsStmt(void)
+{
+ READ_LOCALS(CreateTableAsStmt);
+
+ READ_NODE_FIELD(query);
+ READ_NODE_FIELD(into);
+ READ_ENUM_FIELD(objtype, ObjectType);
+ READ_BOOL_FIELD(is_select_into);
+ READ_BOOL_FIELD(if_not_exists);
+
+ READ_DONE();
+}
+
+/*
+ * _readAlterTableStmt
+ */
+static AlterTableStmt*
+_readAlterTableStmt(void)
+{
+ READ_LOCALS(AlterTableStmt);
+
+ READ_NODE_FIELD(relation);
+ READ_NODE_FIELD(cmds);
+ READ_ENUM_FIELD(objtype, ObjectType);
+ READ_BOOL_FIELD(missing_ok);
+
+ READ_DONE();
+}
+
+/*
+ * _readAlterTableCmd
+ */
+static AlterTableCmd*
+_readAlterTableCmd(void)
+{
+ READ_LOCALS(AlterTableCmd);
+
+ READ_ENUM_FIELD(subtype, AlterTableType);
+ READ_STRING_FIELD(name);
+ READ_INT_FIELD(num);
+ READ_NODE_FIELD(newowner);
+ READ_NODE_FIELD(def);
+ READ_ENUM_FIELD(behavior, DropBehavior);
+ READ_BOOL_FIELD(missing_ok);
+
+ READ_DONE();
+}
+
+/*
+ * _readDropStmt
+ */
+static DropStmt*
+_readDropStmt(void)
+{
+ READ_LOCALS(DropStmt);
+
+ READ_NODE_FIELD(objects);
+ READ_ENUM_FIELD(removeType, ObjectType);
+ READ_ENUM_FIELD(behavior, DropBehavior);
+ READ_BOOL_FIELD(missing_ok);
+ READ_BOOL_FIELD(concurrent);
+
+ READ_DONE();
+}
+
+/*
+ * _readCreateFunctionStmt
+ */
+static CreateFunctionStmt*
+_readCreateFunctionStmt(void)
+{
+ READ_LOCALS(CreateFunctionStmt);
+
+ READ_BOOL_FIELD(is_procedure);
+ READ_BOOL_FIELD(replace);
+ READ_NODE_FIELD(funcname);
+ READ_NODE_FIELD(parameters);
+ READ_NODE_FIELD(returnType);
+ READ_NODE_FIELD(options);
+ READ_NODE_FIELD(sql_body);
+
+ READ_DONE();
+}
+
+/*
+ * _readFunctionParameter
+ */
+static FunctionParameter*
+_readFunctionParameter(void)
+{
+ READ_LOCALS(FunctionParameter);
+
+ READ_STRING_FIELD(name);
+ READ_NODE_FIELD(argType);
+ READ_ENUM_FIELD(mode, FunctionParameterMode);
+ READ_NODE_FIELD(defexpr);
+
+ READ_DONE();
+}
+
+/*
+ * _readAlterFunctionStmt
+ */
+static AlterFunctionStmt*
+_readAlterFunctionStmt(void)
+{
+ READ_LOCALS(AlterFunctionStmt);
+
+ READ_ENUM_FIELD(objtype, ObjectType);
+ READ_NODE_FIELD(func);
+ READ_NODE_FIELD(actions);
+
+ READ_DONE();
+}
+
+/*
+ * _readRoleSpec
+ */
+static RoleSpec*
+_readRoleSpec(void)
+{
+ READ_LOCALS(RoleSpec);
+
+ READ_ENUM_FIELD(type, RoleSpecType);
+ READ_STRING_FIELD(rolename);
+ READ_INT_FIELD(location);
+
+ READ_DONE();
+}
+
+/*
+ * _readColumnDef
+ */
+static ColumnDef*
+_readColumnDef(void)
+{
+ READ_LOCALS(ColumnDef);
+
+ READ_STRING_FIELD(colname);
+ READ_NODE_FIELD(typeName);
+ READ_STRING_FIELD(compression);
+ READ_INT_FIELD(inhcount);
+ READ_BOOL_FIELD(is_local);
+ READ_BOOL_FIELD(is_not_null);
+ READ_BOOL_FIELD(is_from_type);
+ READ_CHAR_FIELD(storage);
+ READ_NODE_FIELD(raw_default);
+ READ_NODE_FIELD(cooked_default);
+ READ_CHAR_FIELD(identity);
+ READ_NODE_FIELD(identitySequence);
+ READ_CHAR_FIELD(generated);
+ READ_NODE_FIELD(collClause);
+ READ_OID_FIELD(collOid);
+ READ_NODE_FIELD(constraints);
+ READ_NODE_FIELD(fdwoptions);
+ READ_LOCATION_FIELD(location);
+
+ READ_DONE();
+}
+
+/*
+ * _readPartitionSpec
+ */
+static PartitionSpec*
+_readPartitionSpec(void)
+{
+ READ_LOCALS(PartitionSpec);
+
+ READ_STRING_FIELD(strategy);
+ READ_NODE_FIELD(partParams);
+ READ_LOCATION_FIELD(location);
+
+ READ_DONE();
+}
+
+/*
+ * _readPartitionElem
+ */
+static PartitionElem*
+_readPartitionElem(void)
+{
+ READ_LOCALS(PartitionElem);
+
+ READ_STRING_FIELD(name);
+ READ_NODE_FIELD(expr);
+ READ_NODE_FIELD(collation);
+ READ_NODE_FIELD(opclass);
+ READ_LOCATION_FIELD(location);
+
+ READ_DONE();
+}
+
+/*
+ * _readTypeName
+ */
+static TypeName*
+_readTypeName(void)
+{
+ READ_LOCALS(TypeName);
+
+ READ_NODE_FIELD(names);
+ READ_OID_FIELD(typeOid);
+ READ_BOOL_FIELD(setof);
+ READ_BOOL_FIELD(pct_type);
+ READ_NODE_FIELD(typmods);
+ READ_INT_FIELD(typemod);
+ READ_NODE_FIELD(arrayBounds);
+ READ_LOCATION_FIELD(location);
+
+ READ_DONE();
+}
+
+/*
+ * _readConstraint
+ */
+static Constraint*
+_readConstraint(void)
+{
+ READ_LOCALS(Constraint);
+ READ_STRING_FIELD(conname);
+ READ_BOOL_FIELD(deferrable);
+ READ_BOOL_FIELD(initdeferred);
+ READ_LOCATION_FIELD(location);
+
+ /* read :contype CONTYPE */
+ token = pg_strtok(&length);
+ token = pg_strtok(&length);
+
+#define MATCH(tokname, namelen) \
+ (length == namelen && memcmp(token, tokname, namelen) == 0)
+
+ if (MATCH("NULL", 4))
+ {
+ local_node->contype = CONSTR_NULL;
+ }
+ else if (MATCH("NOT_NULL", 8))
+ {
+ local_node->contype = CONSTR_NOTNULL;
+ }
+ else if (MATCH("DEFAULT", 7))
+ {
+ local_node->contype = CONSTR_DEFAULT;
+ READ_NODE_FIELD(raw_expr);
+ READ_STRING_FIELD(cooked_expr);
+ }
+ else if (MATCH("IDENTITY", 8))
+ {
+ local_node->contype = CONSTR_IDENTITY;
+ READ_NODE_FIELD(raw_expr);
+ READ_STRING_FIELD(cooked_expr);
+ READ_CHAR_FIELD(generated_when);
+ }
+ else if (MATCH("GENERATED", 9))
+ {
+ local_node->contype = CONSTR_GENERATED;
+ READ_NODE_FIELD(raw_expr);
+ READ_STRING_FIELD(cooked_expr);
+ READ_CHAR_FIELD(generated_when);
+ }
+ else if (MATCH("CHECK", 5))
+ {
+ local_node->contype = CONSTR_CHECK;
+ READ_BOOL_FIELD(is_no_inherit);
+ READ_NODE_FIELD(raw_expr);
+ READ_STRING_FIELD(cooked_expr);
+ }
+ else if (MATCH("PRIMARY_KEY", 11))
+ {
+ local_node->contype = CONSTR_PRIMARY;
+ READ_NODE_FIELD(keys);
+ READ_NODE_FIELD(including);
+ READ_NODE_FIELD(options);
+ READ_STRING_FIELD(indexname);
+ READ_STRING_FIELD(indexspace);
+ READ_BOOL_FIELD(reset_default_tblspc);
+ /* access_method and where_clause not currently used */
+ }
+ else if (MATCH("UNIQUE", 6))
+ {
+ local_node->contype = CONSTR_UNIQUE;
+ READ_BOOL_FIELD(nulls_not_distinct);
+ READ_NODE_FIELD(keys);
+ READ_NODE_FIELD(including);
+ READ_NODE_FIELD(options);
+ READ_STRING_FIELD(indexname);
+ READ_STRING_FIELD(indexspace);
+ READ_BOOL_FIELD(reset_default_tblspc);
+ /* access_method and where_clause not currently used */
+ }
+ else if (MATCH("EXCLUSION", 9))
+ {
+ local_node->contype = CONSTR_EXCLUSION;
+ READ_NODE_FIELD(exclusions);
+ READ_NODE_FIELD(including);
+ READ_NODE_FIELD(options);
+ READ_STRING_FIELD(indexname);
+ READ_STRING_FIELD(indexspace);
+ READ_BOOL_FIELD(reset_default_tblspc);
+ READ_STRING_FIELD(access_method);
+ READ_NODE_FIELD(where_clause);
+ }
+ else if (MATCH("FOREIGN_KEY", 11))
+ {
+ local_node->contype = CONSTR_FOREIGN;
+ READ_NODE_FIELD(pktable);
+ READ_NODE_FIELD(fk_attrs);
+ READ_NODE_FIELD(pk_attrs);
+ READ_CHAR_FIELD(fk_matchtype);
+ READ_CHAR_FIELD(fk_upd_action);
+ READ_CHAR_FIELD(fk_del_action);
+ READ_NODE_FIELD(fk_del_set_cols);
+ READ_NODE_FIELD(old_conpfeqop);
+ READ_OID_FIELD(old_pktable_oid);
+ READ_BOOL_FIELD(skip_validation);
+ READ_BOOL_FIELD(initially_valid);
+ }
+ else if (MATCH("ATTR_DEFERRABLE", 15))
+ {
+ local_node->contype = CONSTR_ATTR_DEFERRABLE;
+ }
+ else if (MATCH("ATTR_NOT_DEFERRABLE", 19))
+ {
+ local_node->contype = CONSTR_ATTR_NOT_DEFERRABLE;
+ }
+ else if (MATCH("ATTR_DEFERRED", 13))
+ {
+ local_node->contype = CONSTR_ATTR_DEFERRED;
+ }
+ else if (MATCH("ATTR_IMMEDIATE", 14))
+ {
+ local_node->contype = CONSTR_ATTR_IMMEDIATE;
+ }
+
+ READ_DONE();
+}
+
+/*
+ * _readIndexStmt
+ */
+static IndexStmt*
+_readIndexStmt(void)
+{
+ READ_LOCALS(IndexStmt);
+
+ READ_STRING_FIELD(idxname);
+ READ_NODE_FIELD(relation);
+ READ_STRING_FIELD(accessMethod);
+ READ_STRING_FIELD(tableSpace);
+ READ_NODE_FIELD(indexParams);
+ READ_NODE_FIELD(indexIncludingParams);
+ READ_NODE_FIELD(options);
+ READ_NODE_FIELD(whereClause);
+ READ_NODE_FIELD(excludeOpNames);
+ READ_STRING_FIELD(idxcomment);
+ READ_OID_FIELD(indexOid);
+ READ_OID_FIELD(oldNode);
+ READ_UINT_FIELD(oldCreateSubid);
+ READ_UINT_FIELD(oldFirstRelfilenodeSubid);
+ READ_BOOL_FIELD(unique);
+ READ_BOOL_FIELD(nulls_not_distinct);
+ READ_BOOL_FIELD(primary);
+ READ_BOOL_FIELD(isconstraint);
+ READ_BOOL_FIELD(deferrable);
+ READ_BOOL_FIELD(initdeferred);
+ READ_BOOL_FIELD(transformed);
+ READ_BOOL_FIELD(concurrent);
+ READ_BOOL_FIELD(if_not_exists);
+ READ_BOOL_FIELD(reset_default_tblspc);
+
+ READ_DONE();
+}
+
+/*
+ * _readIndexElem
+ */
+static IndexElem*
+_readIndexElem(void)
+{
+ READ_LOCALS(IndexElem);
+
+ READ_STRING_FIELD(name);
+ READ_NODE_FIELD(expr);
+ READ_STRING_FIELD(indexcolname);
+ READ_NODE_FIELD(collation);
+ READ_NODE_FIELD(opclass);
+ READ_NODE_FIELD(opclassopts);
+ READ_ENUM_FIELD(ordering, SortByDir);
+ READ_ENUM_FIELD(nulls_ordering, SortByNulls);
+
+ READ_DONE();
+}
+
/*
* parseNodeString
*
@@ -3235,6 +3655,38 @@ parseNodeString(void)
return_value = _readJsonTableParent();
else if (MATCH("JSONTABLESIBLING", 16))
return_value = _readJsonTableSibling();
+ else if (MATCH("CREATESTMT", 10))
+ return_value = _readCreateStmt();
+ else if (MATCH("CREATETABLEASSTMT", 17))
+ return_value = _readCreateTableAsStmt();
+ else if (MATCH("ALTERTABLESTMT", 14))
+ return_value = _readAlterTableStmt();
+ else if (MATCH("ALTERTABLECMD", 13))
+ return_value = _readAlterTableCmd();
+ else if (MATCH("DROPSTMT", 8))
+ return_value = _readDropStmt();
+ else if (MATCH("CREATEFUNCTIONSTMT", 18))
+ return_value = _readCreateFunctionStmt();
+ else if (MATCH("FUNCTIONPARAMETER", 17))
+ return_value = _readFunctionParameter();
+ else if (MATCH("ALTERFUNCTIONSTMT", 17))
+ return_value = _readAlterFunctionStmt();
+ else if (MATCH("ROLESPEC", 8))
+ return_value = _readRoleSpec();
+ else if (MATCH("COLUMNDEF", 9))
+ return_value = _readColumnDef();
+ else if (MATCH("TYPENAME", 8))
+ return_value = _readTypeName();
+ else if (MATCH("PARTITIONELEM", 13))
+ return_value = _readPartitionElem();
+ else if (MATCH("PARTITIONSPEC", 13))
+ return_value = _readPartitionSpec();
+ else if (MATCH("CONSTRAINT", 10))
+ return_value = _readConstraint();
+ else if (MATCH("INDEXSTMT", 9))
+ return_value = _readIndexStmt();
+ else if (MATCH("INDEXELEM", 9))
+ return_value = _readIndexElem();
else
{
elog(ERROR, "badly formatted node string \"%.32s\"...", token);
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index c61b3564a7..da72f41921 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -351,7 +351,7 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
-static void apply_execute_sql_command(const char *cmdstr,
+static void apply_execute_sql_command(const char *message,
const char* role,
const char* search_path,
bool isTopLevel);
@@ -2482,130 +2482,70 @@ execute_sql_command_error_cb(void *arg)
}
/*
- * Execute an SQL command. This can be multiple queries.
- * This is modified based on pglogical_execute_sql_command().
+ * Preprocess certain DDL commands before apply
+ * -Remove data population for table creation
+ * -Enable missing_ok for drop stmt
+ * -Disallow table rewrites using volatile functions
*/
static void
-apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
- bool isTopLevel)
+preprocess_ddl(RawStmt *command, char **schemaname, char **relname, bool *is_partitioned_table)
{
- const char *save_debug_query_string = debug_query_string;
- List *parsetree_list;
- ListCell *parsetree_item;
- MemoryContext oldcontext;
- ErrorContextCallback errcallback;
- int save_nestlevel;
-
- /*
- * Switch to appropriate context for constructing parsetrees.
- */
- oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
- begin_replication_step();
-
- /*
- * Set the current role to the user that executed the command on the
- * publication server.
- * Set the current search_path to the search_path on the publication
- * server when the command was executed.
- */
- save_nestlevel = NewGUCNestLevel();
- SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
- SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
-
- errcallback.callback = execute_sql_command_error_cb;
- errcallback.arg = (char *) cmdstr;
- errcallback.previous = error_context_stack;
- error_context_stack = &errcallback;
-
- debug_query_string = cmdstr;
-
- parsetree_list = pg_parse_query(cmdstr);
-
- /*
- * Do a limited amount of safety checking against CONCURRENTLY commands
- * executed in situations where they aren't allowed. The sender side should
- * provide protection, but better be safe than sorry.
- */
- isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
-
- /*
- * Switch back to transaction context to enter the loop.
- */
- MemoryContextSwitchTo(oldcontext);
-
- foreach(parsetree_item, parsetree_list)
+ switch(nodeTag(command->stmt))
{
- List *plantree_list;
- List *querytree_list;
- RawStmt *command = (RawStmt *) lfirst(parsetree_item);
- CommandTag commandTag;
- MemoryContext per_parsetree_context = NULL;
- Portal portal;
- DestReceiver *receiver;
- bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
- char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
- bool is_partitioned_table = false;
-
- commandTag = CreateCommandTag((Node *)command);
-
- /* The following DDL commands need special handling */
-
- /*
- * Remember the schemaname and relname if the cmd is going to create a table
- * because we will need them for some post-processing after we
- * execute the stmt. At that point, command->stmt may have been freeed up.
- */
- if (commandTag == CMDTAG_CREATE_TABLE)
+ case T_CreateStmt:
{
+ /*
+ * Remember the schemaname and relname if the cmd is going to create a table
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, command->stmt may have been freeed up.
+ */
CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
if (cstmt->inhRelations != NIL || cstmt->partspec != NULL)
- is_partitioned_table = true;
+ *is_partitioned_table = true;
+
+ break;
}
- else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ case T_CreateTableAsStmt:
{
CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
if (castmt->objtype == OBJECT_TABLE)
{
RangeVar *rv = castmt->into->rel;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
/*
- * Force skipping data population to avoid data inconsistency.
- * Data should be replicated from the publisher instead.
- */
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
castmt->into->skipData = true;
}
+ break;
}
/* SELECT INTO */
- else if (commandTag == CMDTAG_SELECT)
+ case T_SelectStmt:
{
SelectStmt *sstmt = (SelectStmt *) command->stmt;
if (sstmt->intoClause != NULL)
{
RangeVar *rv = sstmt->intoClause->rel;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
/*
- * Force skipping data population to avoid data inconsistency.
- * Data should be replicated from the publisher instead.
- */
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
sstmt->intoClause->skipData = true;
}
+ break;
}
- /*
- * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
- * Until we support logical replication of table rewrite, see ATRewriteTables()
- * for details on table rewrite.
- */
- else if (commandTag == CMDTAG_ALTER_TABLE)
+ case T_AlterTableStmt:
{
AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
ListCell *lc;
@@ -2633,14 +2573,151 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
if (contain_volatile_functions(expr))
{
elog(ERROR,
- "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
- cmdstr);
+ "Do not support replication of DDL statement that rewrites table using volatile functions");
}
}
}
}
}
+ break;
+ }
+ case T_DropStmt:
+ {
+ DropStmt *dstmt = (DropStmt *) command->stmt;
+ dstmt->missing_ok = true;
+ break;
+ }
+ default:
+ break;
+ }
+}
+
+/*
+* Table created by DDL replication (database level) is automatically
+* added to the subscription here.
+*
+* Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+* command to set the relstate to SUBREL_STATE_INIT so DML changes on this
+* new table can be replicated without having to manually run
+* "alter subscription ... refresh publication"
+*/
+static void
+handle_create_table(char* relname, char* schemaname, bool is_partitioned_table)
+{
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ bool subscribe_table = true;
+
+ if (is_partitioned_table)
+ {
+ Relation rel = RelationIdGetRelation(relid);
+ char *table_name = RelationGetRelationName(rel);
+ char *schema_name = get_namespace_name(RelationGetNamespace(rel));
+ /*
+ * Connect to the source DB and check whehter the partitioned table should be subscribed.
+ * Because it depends on the setting of publish_via_partition_root, which the subscription
+ * doesn't know.
+ */
+ subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
+ RelationClose(rel);
+ }
+
+ if (subscribe_table)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
}
+ }
+}
+
+/*
+ * Deserialize a ddlmessage into a raw parsetree by calling stringToNode() and
+ * then execute it.
+ *
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *message, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ const char *cmdstr = "";
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+ RawStmt *command = makeNode(RawStmt);
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ //SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) message;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = message;
+
+ command->stmt = stringToNode(message);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ //isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ if (command->stmt)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ CommandTag commandTag = CreateCommandTag((Node *)command);
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ bool is_partitioned_table = false;
+
+ preprocess_ddl(command, &schemaname, &relname, &is_partitioned_table);
/*
* Set up a snapshot if parse analysis/planning will need one.
@@ -2656,23 +2733,10 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
*
* Switch to appropriate context for constructing query and plan trees
* (these can't be in the transaction context, as that will get reset
- * when the command is COMMIT/ROLLBACK). If we have multiple
- * parsetrees, we use a separate context for each one, so that we can
- * free that memory before moving on to the next one. But for the
- * last (or only) parsetree, just use MessageContext, which will be
- * reset shortly after completion anyway. In event of an error, the
- * per_parsetree_context will be deleted when MessageContext is reset.
+ * when the command is COMMIT/ROLLBACK).
*/
- if (lnext(parsetree_list, parsetree_item) != NULL)
- {
- per_parsetree_context =
- AllocSetContextCreate(MessageContext,
- "per-parsetree message context",
- ALLOCSET_DEFAULT_SIZES);
- oldcontext = MemoryContextSwitchTo(per_parsetree_context);
- }
- else
- oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
querytree_list = pg_analyze_and_rewrite_fixedparams(
command,
@@ -2735,63 +2799,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
CommandCounterIncrement();
- /*
- * Table created by DDL replication (database level) is automatically
- * added to the subscription here.
- *
- * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
- * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
- * new table can be replicated without having to manually run
- * "alter subscription ... refresh publication"
- */
- if (relname != NULL)
- {
- Oid relid;
- Oid relnamespace = InvalidOid;
-
- if (schemaname != NULL)
- relnamespace = get_namespace_oid(schemaname, false);
- if (relnamespace != InvalidOid)
- relid = get_relname_relid(relname, relnamespace);
- else
- {
- /*
- * Try to resolve unqualified relname.
- * Notice we have set the search_path to the original search_path on the publisher
- * at the beginning of this function.
- */
- relid = RelnameGetRelid(relname);
- }
-
- if (relid != InvalidOid)
- {
- bool subscribe_table = true;
-
- if (is_partitioned_table)
- {
- Relation rel = RelationIdGetRelation(relid);
- char *table_name = RelationGetRelationName(rel);
- char *schema_name = get_namespace_name(RelationGetNamespace(rel));
- /*
- * Connect to the source DB and check whehter the partitioned table should be subscribed.
- * Because it depends on the setting of publish_via_partition_root, which the subscription
- * doesn't know.
- */
- subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
- RelationClose(rel);
- }
-
- if (subscribe_table)
- {
- AddSubscriptionRelState(MySubscription->oid, relid,
- SUBREL_STATE_INIT,
- InvalidXLogRecPtr);
- ereport(DEBUG1,
- (errmsg_internal("table \"%s\" added to subscription \"%s\"",
- relname, MySubscription->name)));
- }
- }
- }
+ if (relname)
+ handle_create_table(relname, schemaname, is_partitioned_table);
}
/*
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 08278c945c..5f41d66604 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1132,10 +1132,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
if (ddl_need_xlog(InvalidOid, true))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
GetUserId(),
- queryString,
- strlen(queryString));
+ parsetree_str,
+ strlen(parsetree_str));
}
break;
@@ -1172,10 +1173,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
if (ddl_need_xlog(InvalidOid, true))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
GetUserId(),
- queryString,
- strlen(queryString));
+ parsetree_str,
+ strlen(parsetree_str));
}
default:
break;
@@ -1191,16 +1193,26 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
case T_AlterTableStmt:
case T_IndexStmt:
+ break;
+ /*
+ * Rename of objects other than table is only allowed in database level
+ * replication.
+ * Rename of table is allowed in both table level and database level
+ * replication.
+ */
case T_RenameStmt:
{
RenameStmt *stmt = (RenameStmt *) parsetree;
- if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
+ if(!stmt->relation && ddl_need_xlog(InvalidOid, true))
+ {
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
- GetUserId(),
- queryString,
- strlen(queryString));
+ GetUserId(),
+ parsetree_str,
+ strlen(parsetree_str));
}
+ break;
}
case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
break;
@@ -1218,11 +1230,9 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/* Drop of sequence is by logical replication of sequences separately */
case OBJECT_SEQUENCE:
break;
- /* Drop of other objects are allowed in Database level DDL replication only */
+ /* Drop of VIEW and MATVIEW are allowed in Database level DDL replication only */
case OBJECT_VIEW:
case OBJECT_MATVIEW:
- case OBJECT_FOREIGN_TABLE:
- default:
/*
* Log these DropStmt for logical replication if
* there is any FOR ALL TABLES publication with pubddl_database on.
@@ -1231,12 +1241,15 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
if (ddl_need_xlog(InvalidOid, true))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
GetUserId(),
- queryString,
- strlen(queryString));
+ parsetree_str,
+ strlen(parsetree_str));
}
break;
+ default:
+ break;
}
}
/*
@@ -1539,10 +1552,11 @@ ProcessUtilitySlow(ParseState *pstate,
ddl_need_xlog(relid, false))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
GetUserId(),
- queryString,
- strlen(queryString));
+ parsetree_str,
+ strlen(parsetree_str));
}
/* ... and do it */
@@ -1774,10 +1788,11 @@ ProcessUtilitySlow(ParseState *pstate,
ddl_need_xlog(relid, false))
{
const char* prefix = "";
+ char* parsetree_str = nodeToString(parsetree);
LogLogicalDDLMessage(prefix,
GetUserId(),
- queryString,
- strlen(queryString));
+ parsetree_str,
+ strlen(parsetree_str));
}
address =
--
2.32.0
[application/octet-stream] 0011-Remove-non-transactional-ddl-message-decoding-becaus.patch (34.7K, ../../CAAD30ULe-cZTELQJbcAahsOFoUO-ftMxorwBTmj7uYK=_=mwxg@mail.gmail.com/13-0011-Remove-non-transactional-ddl-message-decoding-becaus.patch)
download | inline diff:
From b3dfe88f330f0cefbb4e0612d93e7182d008f474 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 24 May 2022 15:37:34 +0000
Subject: [PATCH 11/12] Remove non-transactional ddl message decoding because
the use case is unclear and it has unresolved issues under concurrency.
---
.../test_decoding/expected/ddlmessages.out | 42 ++++++------
contrib/test_decoding/test_decoding.c | 25 +++-----
.../access/rmgrdesc/logicalddlmsgdesc.c | 3 +-
src/backend/commands/tablecmds.c | 8 +--
src/backend/replication/logical/ddlmessage.c | 12 ++--
src/backend/replication/logical/decode.c | 13 +---
src/backend/replication/logical/logical.c | 26 ++++----
src/backend/replication/logical/proto.c | 12 +---
.../replication/logical/reorderbuffer.c | 64 ++++++-------------
src/backend/replication/logical/worker.c | 3 +-
src/backend/replication/pgoutput/pgoutput.c | 17 ++---
src/backend/tcop/utility.c | 24 ++-----
src/include/replication/ddlmessage.h | 3 +-
src/include/replication/logicalproto.h | 4 +-
src/include/replication/output_plugin.h | 2 -
src/include/replication/reorderbuffer.h | 6 +-
16 files changed, 87 insertions(+), 177 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 823029d03d..56cbafa4cf 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -24,14 +24,14 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
-- Test logging DDL in function
@@ -52,20 +52,20 @@ SELECT func_ddl ('tab_from_func');
(1 row)
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
- RETURNS VOID AS $$ +
- BEGIN +
- execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
- execute format('ALTER TABLE %I ADD c3 int', tname); +
- execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
- execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
- END; +
- $$ +
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
LANGUAGE plpgsql;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
BEGIN
table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index a44e1f79e3..eb3dd76782 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -78,7 +78,7 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix,
+ const char *prefix,
const char *role, const char *search_path,
Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
@@ -123,7 +123,7 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix,
+ const char *prefix,
const char *role, const char *search_path,
Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
@@ -771,13 +771,13 @@ pg_decode_message(LogicalDecodingContext *ctx,
static void
pg_decode_ddlmessage(LogicalDecodingContext *ctx,
- ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ ReorderBufferTXN *txn, XLogRecPtr lsn,
const char *prefix, const char *role, const char *search_path,
Size sz, const char *message)
{
OutputPluginPrepareWrite(ctx, true);
- appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
- transactional, prefix, role, search_path, sz);
+ appendStringInfo(ctx->out, "DDL message: prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
OutputPluginWrite(ctx, true);
}
@@ -989,23 +989,14 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
*/
static void
pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
- ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ ReorderBufferTXN *txn, XLogRecPtr lsn,
const char *prefix, const char * role, const char * search_path,
Size sz, const char *message)
{
OutputPluginPrepareWrite(ctx, true);
- if (transactional)
- {
- appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
- transactional, prefix, role, search_path, sz);
- }
- else
- {
- appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
- transactional, prefix, role, search_path, sz);
- appendBinaryStringInfo(ctx->out, message, sz);
- }
+ appendStringInfo(ctx->out, "streaming DDL message: prefix: %s role: %s, search_path: %s, sz: %zu",
+ prefix, role, search_path, sz);
OutputPluginWrite(ctx, true);
}
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
index 7a352d540a..0aaebf8a08 100644
--- a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -32,8 +32,7 @@ logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
Assert(prefix[xlrec->prefix_size] != '\0');
- appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
- xlrec->transactional ? "transactional" : "non-transactional",
+ appendStringInfo(buf, "prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
prefix, role, search_path, xlrec->message_size);
/* Write message payload as a series of hex bytes */
for (int cnt = 0; cnt < xlrec->message_size; cnt++)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 65386f2641..94e350b80d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1516,13 +1516,11 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
/* Log the Drop command for logical replication */
if (ddlxlog)
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
pstate->p_sourcetext,
- strlen(pstate->p_sourcetext),
- transactional);
+ strlen(pstate->p_sourcetext));
}
performMultipleDeletions(objects, drop->behavior, flags);
@@ -3876,13 +3874,11 @@ RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
if (ddlxlog &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
pstate->p_sourcetext,
- strlen(pstate->p_sourcetext),
- transactional);
+ strlen(pstate->p_sourcetext));
}
/* Do the work */
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
index f93573079a..5f89afec49 100644
--- a/src/backend/replication/logical/ddlmessage.c
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -47,7 +47,7 @@
*/
XLogRecPtr
LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
- size_t size, bool transactional)
+ size_t size)
{
xl_logical_ddl_message xlrec;
const char *role;
@@ -55,16 +55,12 @@ LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
role = GetUserNameFromId(roleoid, false);
/*
- * Force xid to be allocated if we're emitting a transactional message.
+ * Force xid to be allocated since we're emitting a transactional message.
*/
- if (transactional)
- {
- Assert(IsTransactionState());
- GetCurrentTransactionId();
- }
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
xlrec.dbId = MyDatabaseId;
- xlrec.transactional = transactional;
/* trailing zero is critical; see logicalddlmsg_desc */
xlrec.prefix_size = strlen(prefix) + 1;
xlrec.role_size = strlen(role) + 1;
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 034c7f2413..ce5b595326 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -615,7 +615,6 @@ logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
TransactionId xid = XLogRecGetXid(r);
uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
RepOriginId origin_id = XLogRecGetOrigin(r);
- Snapshot snapshot;
xl_logical_ddl_message *message;
if (info != XLOG_LOGICAL_DDL_MESSAGE)
@@ -637,17 +636,7 @@ logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
FilterByOrigin(ctx, origin_id))
return;
- if (message->transactional &&
- !SnapBuildProcessChange(builder, xid, buf->origptr))
- return;
- else if (!message->transactional &&
- (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
- SnapBuildXactNeedsSkip(builder, buf->origptr)))
- return;
-
- snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
- ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
- message->transactional,
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
message->message,
/* first part of message is prefix */
message->message + message->prefix_size,
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 3004f02433..c02ea6fb99 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -74,8 +74,8 @@ static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role, const char *search_path,
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
Size message_size, const char *message);
/* streaming callbacks */
@@ -95,8 +95,8 @@ static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *tx
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role, const char *search_path,
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -1233,10 +1233,9 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role,
- const char *search_path, Size message_size,
- const char *message)
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
LogicalErrorCallbackState state;
@@ -1262,7 +1261,7 @@ ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix,
role, search_path, message_size, message);
/* Pop the error context stack */
@@ -1586,10 +1585,9 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role,
- const char* search_path, Size message_size,
- const char *message)
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char* search_path,
+ Size message_size, const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
LogicalErrorCallbackState state;
@@ -1619,7 +1617,7 @@ stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix,
role, search_path, message_size, message);
/* Pop the error context stack */
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 3cfa94dd8c..2072207647 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -670,16 +670,12 @@ logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
const char **prefix,
const char **role,
const char **search_path,
- bool *transactional,
Size *sz)
{
- uint8 flags;
const char *msg;
//TODO double check when do we need to get TransactionId.
- flags = pq_getmsgint(in, 1);
- *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
*lsn = pq_getmsgint64(in);
*prefix = pq_getmsgstring(in);
*role = pq_getmsgstring(in);
@@ -695,22 +691,16 @@ logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
*/
void
logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message)
{
- uint8 flags = 0;
pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
- /* encode and send message flags */
- if (transactional)
- flags |= MESSAGE_TRANSACTIONAL;
-
/* transaction ID (if not valid, we're not streaming) */
if (TransactionIdIsValid(xid))
pq_sendint32(out, xid);
- pq_sendint8(out, flags);
pq_sendint64(out, lsn);
pq_sendstring(out, prefix);
pq_sendstring(out, role);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index ca01336604..f5a3247348 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -881,61 +881,33 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
- * A transactional DDL message is queued to be processed upon commit and a
- * non-transactional DDL message gets processed immediately.
+ * A transactional DDL message is queued to be processed upon commit
*/
void
ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
- Snapshot snapshot, XLogRecPtr lsn,
- bool transactional, const char *prefix,
+ XLogRecPtr lsn, const char *prefix,
const char *role, const char *search_path,
Size message_size, const char *message)
{
- if (transactional)
- {
- MemoryContext oldcontext;
- ReorderBufferChange *change;
-
- Assert(xid != InvalidTransactionId);
-
- oldcontext = MemoryContextSwitchTo(rb->context);
-
- change = ReorderBufferGetChange(rb);
- change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
- change->data.ddlmsg.prefix = pstrdup(prefix);
- change->data.ddlmsg.role = pstrdup(role);
- change->data.ddlmsg.search_path = pstrdup(search_path);
- change->data.ddlmsg.message_size = message_size;
- change->data.ddlmsg.message = palloc(message_size);
- memcpy(change->data.ddlmsg.message, message, message_size);
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
- ReorderBufferQueueChange(rb, xid, lsn, change, false);
+ Assert(xid != InvalidTransactionId);
- MemoryContextSwitchTo(oldcontext);
- }
- else
- {
- ReorderBufferTXN *txn = NULL;
- volatile Snapshot snapshot_now = snapshot;
+ oldcontext = MemoryContextSwitchTo(rb->context);
- if (xid != InvalidTransactionId)
- txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
- /* setup snapshot to allow catalog access */
- SetupHistoricSnapshot(snapshot_now, NULL);
- PG_TRY();
- {
- rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
- TeardownHistoricSnapshot(false);
- }
- PG_CATCH();
- {
- TeardownHistoricSnapshot(true);
- PG_RE_THROW();
- }
- PG_END_TRY();
- }
+ MemoryContextSwitchTo(oldcontext);
}
/*
@@ -2037,14 +2009,14 @@ ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferChange *change, bool streaming)
{
if (streaming)
- rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ rb->stream_ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
change->data.ddlmsg.role,
change->data.ddlmsg.search_path,
change->data.ddlmsg.message_size,
change->data.ddlmsg.message);
else
- rb->ddlmessage(rb, txn, change->lsn, true,
+ rb->ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
change->data.ddlmsg.role,
change->data.ddlmsg.search_path,
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 610147ca77..c61b3564a7 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2461,14 +2461,13 @@ static void
apply_handle_ddlmessage(StringInfo s)
{
XLogRecPtr lsn;
- bool transactional;
Size sz;
const char *prefix;
const char *role;
const char *search_path;
const char *msg;
- msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &sz);
apply_execute_sql_command(msg, role, search_path, true);
}
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index db221c7e8d..a66367fe7a 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -56,7 +56,7 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
@@ -1705,13 +1705,14 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
+ XLogRecPtr message_lsn,
const char *prefix, const char * role,
const char *search_path, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
ListCell *lc;
+ PGOutputTxnData *txndata;
/* Reload publications if needed before use. */
if (!publications_valid)
@@ -1745,20 +1746,16 @@ pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
* Output BEGIN if we haven't yet. Avoid for non-transactional
* messages.
*/
- if (transactional)
- {
- PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ txndata = (PGOutputTxnData *) txn->output_plugin_private;
- /* Send BEGIN if we haven't yet */
- if (txndata && !txndata->sent_begin_txn)
- pgoutput_send_begin(ctx, txn);
- }
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
OutputPluginPrepareWrite(ctx, true);
logicalrep_write_ddlmessage(ctx->out,
xid,
message_lsn,
- transactional,
prefix,
role,
search_path,
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1abd77a60e..08278c945c 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1131,13 +1131,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
break;
@@ -1173,13 +1171,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
default:
break;
@@ -1199,13 +1195,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
{
RenameStmt *stmt = (RenameStmt *) parsetree;
if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
}
case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
@@ -1236,13 +1230,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
break;
}
@@ -1546,13 +1538,11 @@ ProcessUtilitySlow(ParseState *pstate,
isCompleteQuery &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
/* ... and do it */
@@ -1783,13 +1773,11 @@ ProcessUtilitySlow(ParseState *pstate,
isCompleteQuery &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
address =
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
index 1e8ef22296..0418d16dca 100644
--- a/src/include/replication/ddlmessage.h
+++ b/src/include/replication/ddlmessage.h
@@ -20,7 +20,6 @@
typedef struct xl_logical_ddl_message
{
Oid dbId; /* database Oid emitted from */
- bool transactional; /* is message transactional? */
Size prefix_size; /* length of prefix */
Size role_size; /* length of the role that executes the DDL command */
Size search_path_size; /* length of the search path */
@@ -36,7 +35,7 @@ typedef struct xl_logical_ddl_message
#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
- size_t size, bool transactional);
+ size_t size);
/* RMGR API*/
#define XLOG_LOGICAL_DDL_MESSAGE 0x00
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 28ff562d62..cc0ba60905 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -233,11 +233,11 @@ extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecP
bool transactional, const char *prefix,
Size sz, const char *message);
extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message);
extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
const char **role, const char **search_path,
- bool *transactional, Size *sz);
+ Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 5b1c245b72..444b75bef0 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -94,7 +94,6 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -219,7 +218,6 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index dd89e08efc..cd8c69ceb8 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -445,7 +445,6 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -523,7 +522,6 @@ typedef void (*ReorderBufferStreamDDLMessageCB) (
ReorderBuffer *rb,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -671,8 +669,8 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
-extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, const char *role,
const char *search_path, Size message_size, const char *message);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-13 10:08 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-13 10:08 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Sunday, June 12, 2022 2:46 PM Zheng Li <zhengli10@gmail.com> wrote:
>
> > > > > > I've not looked at these patches in-depth yet but with this
> > > > > > approach, what do you think we can handle the DDL syntax
> > > > > > differences between major versions? DDL syntax or behavior
> > > > > > could be changed by future changes and I think we need to
> > > > > > somehow deal with the differences. For
> > > > >
> > > > > > example, if the user uses logical replication for major
> > > > > > version upgrade, the publisher is older than the subscriber.
> > > > > > We might have to rewrite the DDL before applying to the
> > > > > > subscriber because the DDL executed on the publisher no longer
> > > > > > work on a new PostgreSQL version
> > > > >
> > > > > I don't think we will allow this kind of situation to happen in
> > > > > the first place for backward compatibility. If a DDL no longer
> > > > > works on a new version of PostgreSQL, the user will have to
> > > > > change the application code as well.
> > > > > So even if it happens for
> > > > > whatever reason, we could either 1. fail the apply worker and
> > > > > let the user fix such DDL because they'll have to fix the
> > > > > application code anyway when this happens.
> > > > > 2. add guard rail logic in the apply worker to automatically fix
> > > > > such DDL if possible, knowing the version of the source and
> > > > > target. Similar logic must have been implemented for
> pg_dump/restore/upgrade.
> > > > >
> > > > > > or we might have to add some options to the DDL before the
> > > > > > application in order to keep the same behavior. This seems to
> > > > > > require a different solution from what the patch does for the
> > > > > > problem you mentioned such
> > > > >
> > > > > > as "DDL involving multiple tables where only some tables are
> > > > > > replicated”.
> > > > >
> > > > > First of all, this case can only happen when the customer
> > > > > chooses to only replicate a subset of the tables in a database
> > > > > in which case table level DDL replication is chosen instead of
> > > > > database level DDL replication (where all tables and DDLs are
> > > > > replicated). I think the solution would be:
> > > > > 1. make best effort to detect such DDLs on the publisher and
> > > > > avoid logging of such DDLs in table level DDL replication.
> > > > > 2. apply worker will fail to replay such command due to missing
> > > > > objects if such DDLs didn't get filtered on the publisher for
> > > > > some reason. This should be rare and I think it's OK even if it
> > > > > happens, we'll find out why and fix it.
> > > > >
> > > >
> > > > FWIW, both these cases could be handled with the deparsing
> > > > approach, and the handling related to the drop of multiple tables
> > > > where only a few are published is already done in the last POC
> > > > patch shared by Ajin [1].
> > > >
> > >
> > > Right. So I'm inclined to think that deparsing approach is better
> > > from this point as well as the point mentioned by Álvaro before[1].
> >
> > I agree. One more point about deparsing approach is that it can also
> > help to replicate CREATE TABLE AS/SELECT INTO in a better way.
> >
> > The main idea of replicating the CREATE TABLE AS is that we deprase
> > the CREATE TABLE AS into a simple CREATE TABLE(without subquery)
> > command and WAL log it after creating the table and before writing
> > data into the table and replicate the incoming writes later as normal
> > INSERTs. In this apporach, we don't execute the subquery on subscriber
> > so that don't need to make sure all the objects referenced in the
> > subquery also exists in subscriber. And This approach works for all
> > kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
> >
> > One problem of this approach is that we cannot use the current trigger
> > to deparse or WAL log the CREATE TABLE. Because none of the even
> > trigger is fired after creating the table and before inserting the
> > data. To solve this, one idea is that we could directly add some code
> > at the end of create_ctas_internal() to deparse and WAL log it.
> > Moreover, we could even introduce a new type of event
> > trigger(table_create) which would be fired at the expected timing so
> > that we can use the trigger function to deparse and WAL log. I am not
> > sure which way is better. I temporarily use the second idea which
> > introduce a new type event trigger in the 0003 POC patch.
>
> Hi, I agree that an intermediate structured format (with all objects schema
> qualified) makes it easier to handle syntax differences between the publisher
> and the subscriber. Such structured format is also likely easier to use by other
> logical replication consumers.
> However, to make things more maintainable, would it be better to use the
> existing serialization/deserialization functions in out/readfuncs.c to generate
> the parsetree representation of the DDL command?
>
> It turns out support for DDL commands are mostly missing in out/readfuncs at
> the moment. see the following comment from
> outfuncs.c:
>
> * Output support for raw parsetrees
> * is somewhat incomplete, too; in particular, utility statements are
> * almost entirely unsupported. We try to support everything that can
> * appear in a raw SELECT, though.
>
> So what about adding support for utility statements in out/readfuncs.c so that
> nodeToString()/stringToNode() works on raw parsetrees of utility statements? I
> think there'll be some benefits:
> 1. It's less code compared to introducing brand new
> serialization/deserialization code for DDL commands.
> 2. It's more maintainable since hackers are already familiar with
> out/readfuncs.c.
> 3. Support for utility statements in out/readfuncs.c may turn out to be useful
> for future features that need serialization/deserialization of DDL raw
> parsetrees.
>
> In patch 0012 I explored the above idea by changing the logging format of
> logical ddlmessage to that of the serialization of the raw parsetree of the DDL
> command using nodeToString(), the apply worker deserializes the message
> back to the raw parsetree by calling stringToNode().
> Here are the detailed changes:
> - Adding serialization/deserialization functions in outfuncs.c/readfuncs.c
> for CreateTableStmt, AlterTableStmt, DropStmt, CreateFunctionStmt and
> AlterFunctionStmt.
> - Modified the serialization process to always schema qualify object names,
> this is done by outQualifiedName() and a change in _outRangeVar().
> - Change the input of LogLogicalDDLMessage() to use
> nodeToString(parsetree).
> - Change the apply worker to call stringToNode(ddlmessage) to get the
> raw parsetree back and then directly execute the parsetree.
>
> This patch doesn't introduce any deparsing functionality yet, but it also
> provides the flexibility to edit the command being replayed by directly
> modifying the raw parsetree on the apply worker (e.g. setting the missing_ok
> flag for DropStmt is equivalent to adding "IF EXIST" to the statement).
>
> Thoughts?
Hi,
Thanks for providing this idea.
I looked at the string that is used for replication:
"""
{ALTERTABLESTMT :relation {RANGEVAR :schemaname public :relname foo
:inh true :relpersistence p :alias <> :location 12} :cmds ({ALTERTABLECMD
:subtype 0 :name <> :num 0 :newowner <> :def {COLUMNDEF :colname b
:typeName {TYPENAME :names ("public" "timestamptz") :typeOid 0 :setof
false :pct_type false :typmods <> :typemod -1 :arrayBounds <> :location
29} :compression <> :inhcount 0 :is_local true :is_not_null false
:is_from_type false :storage <> :raw_default <> :cooked_default <>
:identity <> :identitySequence <> :generated <> :collClause <> :collOid 0
:constraints <> :fdwoptions <> :location 27} :behavior 0 :missing_ok
false}) :objtype 41 :missing_ok false}
"""
I think the converted parsetree string includes lots of internal
objects(e.g typeOid/pct_type/objtype/collOid/location/...). These are
unnecessary stuff for replication and we cannot make sure all the internal
stuff are consistent among pub/sub. So I am not sure whether replicating
this string is better.
Besides, replicating the string from nodetostring() means we would need to
deal with the structure difference between the publisher and the
subscriber if any related structure has been changed which seems not good.
IMO, The advantages of the deparsing approach(as implemented in the POC
patch set[1]) are:
1) We can generate a command representation that can be
parsed/processed/transformed arbitrarily by the subscriber using generic
rules it(for example: user can easily replace the schema name in it) while
the results of nodetostring() seems not a standard json string, so I am
not sure can user reuse it without traversing the parsetree again.
2) With event_trigger + deparser, we can filter the unpublished objects
easier. For example: "DROP TABLE table_pub, table_unpub;". We can deparse
it into two commands "DROP TABLE table_pub" and "DROP TABLE table_pub" and
only publish the first one.
3) With deparser, we are able to query the catalog in the deparser to
build a complete command(filled with schemaname...) which user don't need
to do any other work for it. We don't need to force the subscriber to set
the same search_path as the publisher which give user more flexibility.
4) For CREATE TABLE AS, we can separate out the CREATE TABLE part with the
help of deparser and event trigger. This can avoid executing the subquery
on subcriber.
5) For ALTER TABLE command. We might want to filter out the DDL which use
volatile function as discussed in [2]. We can achieve this easier by
extending the deparser to check the functions used. We can even rebuild a
command without unsupported functions to replicate by using deparser.
There may be more cases I am missing as we are still analyzing other DDLs.
And most of these ideas has been implemented in the POC patch[1].
The overhead to maintain the deparser is not a problem as long as we have
a good test mechanism to verify that all DDLs are supported.
Overall, the event_trigger + deparsing approach looks better to me.
[1] https://www.postgresql.org/message-id/OS0PR01MB5716733D6C79D2198E5773CC94A79%40OS0PR01MB5716.jpnprd0...
[2] https://www.postgresql.org/message-id/CAA4eK1JVynFsj%2BmcRWj9sewR2yNUs6LuNxJ0eN-gNJ83oKcUOQ%40mail.g...
Best regards,
Hou zj
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-15 00:14 Zheng Li <zhengli10@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-06-15 00:14 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> Thanks for providing this idea.
>
> I looked at the string that is used for replication:
>
> """
> {ALTERTABLESTMT :relation {RANGEVAR :schemaname public :relname foo
> :inh true :relpersistence p :alias <> :location 12} :cmds ({ALTERTABLECMD
> :subtype 0 :name <> :num 0 :newowner <> :def {COLUMNDEF :colname b
> :typeName {TYPENAME :names ("public" "timestamptz") :typeOid 0 :setof
> false :pct_type false :typmods <> :typemod -1 :arrayBounds <> :location
> 29} :compression <> :inhcount 0 :is_local true :is_not_null false
> :is_from_type false :storage <> :raw_default <> :cooked_default <>
> :identity <> :identitySequence <> :generated <> :collClause <> :collOid 0
> :constraints <> :fdwoptions <> :location 27} :behavior 0 :missing_ok
> false}) :objtype 41 :missing_ok false}
> """
>
> I think the converted parsetree string includes lots of internal
> objects(e.g typeOid/pct_type/objtype/collOid/location/...). These are
> unnecessary stuff for replication and we cannot make sure all the internal
> stuff are consistent among pub/sub. So I am not sure whether replicating
> this string is better.
>
> Besides, replicating the string from nodetostring() means we would need to
> deal with the structure difference between the publisher and the
> subscriber if any related structure has been changed which seems not good.
Yeah, this existing format is not designed to be portable between different
major versions. So it can't directly be used for replication without
serious modification.
> IMO, The advantages of the deparsing approach(as implemented in the POC
> patch set[1]) are:
>
> 1) We can generate a command representation that can be
> parsed/processed/transformed arbitrarily by the subscriber using generic
> rules it(for example: user can easily replace the schema name in it) while
> the results of nodetostring() seems not a standard json string, so I am
> not sure can user reuse it without traversing the parsetree again.
>
> 2) With event_trigger + deparser, we can filter the unpublished objects
> easier. For example: "DROP TABLE table_pub, table_unpub;". We can deparse
> it into two commands "DROP TABLE table_pub" and "DROP TABLE table_pub" and
> only publish the first one.
>
> 3) With deparser, we are able to query the catalog in the deparser to
> build a complete command(filled with schemaname...) which user don't need
> to do any other work for it. We don't need to force the subscriber to set
> the same search_path as the publisher which give user more flexibility.
>
> 4) For CREATE TABLE AS, we can separate out the CREATE TABLE part with the
> help of deparser and event trigger. This can avoid executing the subquery
> on subcriber.
>
> 5) For ALTER TABLE command. We might want to filter out the DDL which use
> volatile function as discussed in [2]. We can achieve this easier by
> extending the deparser to check the functions used. We can even rebuild a
> command without unsupported functions to replicate by using deparser.
>
> There may be more cases I am missing as we are still analyzing other DDLs.
How does the deparser deparses CREATE FUNCTION STATEMENT? Will it schema qualify
objects inside the function definition?
While I agree that the deparser is needed to handle the potential
syntax differences between
the pub/sub, I think it's only relevant for the use cases where only a
subset of tables in the database
are replicated. For other use cases where all tables, functions and
other objects need to be replicated,
(for example, creating a logical replica for major version upgrade)
there won't be any syntax difference to
handle and the schemas are supposed to match exactly between the
pub/sub. In other words the user seeks to create an identical replica
of the source database and the DDLs should be replicated
as is in this case. So I think it's an overkill to use deparser for
such use cases. It also costs more space and
time using deparsing. For example, the following simple ALTER TABLE
command incurs 11 times more space
in the WAL record if we were to use the format from the deparser,
there will also be time and CPU overhead from the deparser.
ALTER TABLE t1 ADD c INT; serach_path: "$user", public
VS
{\"fmt\": \"ALTER TABLE %{identity}D %{subcmds:, }s\", \"subcmds\":
[{\"fmt\": \"ADD COLUMN %{definition}s\", \"type\": \"add column\",
\"definiti
on\": {\"fmt\": \"%{name}I %{coltype}T %{default}s %{not_null}s
%{collation}s\", \"name\": \"c\", \"type\": \"column\", \"coltype\":
{\"typmod\": \"\", \"typarray
\": false, \"typename\": \"int4\", \"schemaname\": \"pg_catalog\"},
\"default\": {\"fmt\": \"DEFAULT %{default}s\", \"present\": false},
\"not_null\": \"\", \"col
lation\": {\"fmt\": \"COLLATE %{name}D\", \"present\": false}}}],
\"identity\": {\"objname\": \"t1\", \"schemaname\": \"public\"}}
So I think it's better to define DDL replication levels [1] to tailor
for the two different use cases. We can use different logging format
based on the DDL replication level. For example,
we can simply log the DDL query string and the search_path for
database level DDL replication. But for table level DDL replication we
need to use the deparser format in order to
handle the potential syntax differences and schema mapping requests.
[1] https://www.postgresql.org/message-id/CAAD30U%2BpVmfKwUKy8cbZOnUXyguJ-uBNejwD75Kyo%3DOjdQGJ9g%40mail...
Thoughts?
With Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-15 04:00 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 2 replies; 449+ messages in thread
From: Amit Kapila @ 2022-06-15 04:00 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
>
>
> While I agree that the deparser is needed to handle the potential
> syntax differences between
> the pub/sub, I think it's only relevant for the use cases where only a
> subset of tables in the database
> are replicated. For other use cases where all tables, functions and
> other objects need to be replicated,
> (for example, creating a logical replica for major version upgrade)
> there won't be any syntax difference to
> handle and the schemas are supposed to match exactly between the
> pub/sub. In other words the user seeks to create an identical replica
> of the source database and the DDLs should be replicated
> as is in this case.
>
I think even for database-level replication we can't assume that
source and target will always have the same data in which case "Create
Table As ..", "Alter Table .. " kind of statements can't be replicated
as it is because that can lead to different results. The other point
is tomorrow we can extend the database level option/syntax to exclude
a few objects (something like [1]) as well in which case we again need
to filter at the publisher level.
>
So I think it's an overkill to use deparser for
> such use cases. It also costs more space and
> time using deparsing. For example, the following simple ALTER TABLE
> command incurs 11 times more space
> in the WAL record if we were to use the format from the deparser,
> there will also be time and CPU overhead from the deparser.
>
...
>
> So I think it's better to define DDL replication levels [1] to tailor
> for the two different use cases. We can use different logging format
> based on the DDL replication level. For example,
> we can simply log the DDL query string and the search_path for
> database level DDL replication. But for table level DDL replication we
> need to use the deparser format in order to
> handle the potential syntax differences and schema mapping requests.
>
I think having different logging formats is worth considering but I am
not sure we can distinguish it for database and table level
replication because of the reasons mentioned above. One thing which
may need a different format is the replication of global objects like
roles, tablespace, etc. but we haven't analyzed them in detail about
those. I feel we may also need a different syntax altogether to
replicate such objects. Also, I think we may want to optimize the
current format in some cases so that the WAL amount could be reduced.
I feel if we think that deparsing is required for this project then
probably at this stage it would be a good idea to explore ways to have
independent ways to test it. One way is to do testing via the logical
replication of DDL (aka via this patch) and the other is to write an
independent test suite as Sawada-San seems to be speculating above
[2]. I am not sure if there is any progress yet on the independent
test suite front yet.
[1] - https://commitfest.postgresql.org/38/3646/
[2] - https://www.postgresql.org/message-id/CAD21AoAX_xiO03hXSY2QfbcKT0XiUvtnzTjy%2BNRJ_EcgBa5B3A%40mail.g...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-15 06:45 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-15 06:45 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wednesday, June 15, 2022 8:14 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> > Thanks for providing this idea.
> >
> > I looked at the string that is used for replication:
> >
> > """
> > {ALTERTABLESTMT :relation {RANGEVAR :schemaname public :relname foo
> > :inh true :relpersistence p :alias <> :location 12} :cmds ({ALTERTABLECMD
> > :subtype 0 :name <> :num 0 :newowner <> :def {COLUMNDEF :colname b
> > :typeName {TYPENAME :names ("public" "timestamptz") :typeOid 0 :setof
> > false :pct_type false :typmods <> :typemod -1 :arrayBounds <> :location
> > 29} :compression <> :inhcount 0 :is_local true :is_not_null false
> > :is_from_type false :storage <> :raw_default <> :cooked_default <>
> > :identity <> :identitySequence <> :generated <> :collClause <> :collOid 0
> > :constraints <> :fdwoptions <> :location 27} :behavior 0 :missing_ok
> > false}) :objtype 41 :missing_ok false}
> > """
> >
> > I think the converted parsetree string includes lots of internal
> > objects(e.g typeOid/pct_type/objtype/collOid/location/...). These are
> > unnecessary stuff for replication and we cannot make sure all the internal
> > stuff are consistent among pub/sub. So I am not sure whether replicating
> > this string is better.
> >
> > Besides, replicating the string from nodetostring() means we would need to
> > deal with the structure difference between the publisher and the
> > subscriber if any related structure has been changed which seems not good.
>
> Yeah, this existing format is not designed to be portable between different
> major versions. So it can't directly be used for replication without
> serious modification.
>
> > IMO, The advantages of the deparsing approach(as implemented in the POC
> > patch set[1]) are:
> >
> > 1) We can generate a command representation that can be
> > parsed/processed/transformed arbitrarily by the subscriber using generic
> > rules it(for example: user can easily replace the schema name in it) while
> > the results of nodetostring() seems not a standard json string, so I am
> > not sure can user reuse it without traversing the parsetree again.
> >
> > 2) With event_trigger + deparser, we can filter the unpublished objects
> > easier. For example: "DROP TABLE table_pub, table_unpub;". We can deparse
> > it into two commands "DROP TABLE table_pub" and "DROP TABLE
> table_pub" and
> > only publish the first one.
> >
> > 3) With deparser, we are able to query the catalog in the deparser to
> > build a complete command(filled with schemaname...) which user don't need
> > to do any other work for it. We don't need to force the subscriber to set
> > the same search_path as the publisher which give user more flexibility.
> >
> > 4) For CREATE TABLE AS, we can separate out the CREATE TABLE part with the
> > help of deparser and event trigger. This can avoid executing the subquery
> > on subcriber.
> >
> > 5) For ALTER TABLE command. We might want to filter out the DDL which use
> > volatile function as discussed in [2]. We can achieve this easier by
> > extending the deparser to check the functions used. We can even rebuild a
> > command without unsupported functions to replicate by using deparser.
> >
> > There may be more cases I am missing as we are still analyzing other DDLs.
>
> How does the deparser deparses CREATE FUNCTION STATEMENT? Will it
> schema qualify
> objects inside the function definition?
The current deparser doesn't schema qualify objects inside the function
source as we won't know the schema of inner objects until the function is
executed. The deparser will only schema qualify the objects around
function declaration Like:
CREATE FUNCTION [public].test_func(i [pg_catalog].int4 ) RETURNS [pg_catalog].int4 LANGUAGE plpgsql
Best regards,
Hou zj
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-17 19:38 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-06-17 19:38 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Jun 15, 2022 at 12:00 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> >
> >
> > While I agree that the deparser is needed to handle the potential
> > syntax differences between
> > the pub/sub, I think it's only relevant for the use cases where only a
> > subset of tables in the database
> > are replicated. For other use cases where all tables, functions and
> > other objects need to be replicated,
> > (for example, creating a logical replica for major version upgrade)
> > there won't be any syntax difference to
> > handle and the schemas are supposed to match exactly between the
> > pub/sub. In other words the user seeks to create an identical replica
> > of the source database and the DDLs should be replicated
> > as is in this case.
> >
>
> I think even for database-level replication we can't assume that
> source and target will always have the same data in which case "Create
> Table As ..", "Alter Table .. " kind of statements can't be replicated
> as it is because that can lead to different results.
"Create Table As .." is already handled by setting the skipData flag of
the statement parsetreee before replay:
/*
* Force skipping data population to avoid data inconsistency.
* Data should be replicated from the publisher instead.
*/
castmt->into->skipData = true;
"Alter Table .. " that rewrites with volatile expressions can also be handled
without any syntax change, by enabling the table rewrite replication and
converting the rewrite inserts to updates. ZJ's patch introduced this solution.
I've also adopted this approach in my latest patch
0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch
> The other point
> is tomorrow we can extend the database level option/syntax to exclude
> a few objects (something like [1]) as well in which case we again need
> to filter at the publisher level
I think for such cases it's not full database replication and we could treat it
as table level DDL replication, i.e. use the the deparser format.
> > So I think it's better to define DDL replication levels [1] to tailor
> > for the two different use cases. We can use different logging format
> > based on the DDL replication level. For example,
> > we can simply log the DDL query string and the search_path for
> > database level DDL replication. But for table level DDL replication we
> > need to use the deparser format in order to
> > handle the potential syntax differences and schema mapping requests.
> >
>
> I think having different logging formats is worth considering but I am
> not sure we can distinguish it for database and table level
> replication because of the reasons mentioned above. One thing which
> may need a different format is the replication of global objects like
> roles, tablespace, etc. but we haven't analyzed them in detail about
> those. I feel we may also need a different syntax altogether to
> replicate such objects.
Yes, global objects are not schema qualified so we probably don't need to
use the deparser format for these. We plan to do some evaluation on replication
of global objects.
> Also, I think we may want to optimize the
> current format in some cases so that the WAL amount could be reduced.
>
> I feel if we think that deparsing is required for this project then
> probably at this stage it would be a good idea to explore ways to have
> independent ways to test it. One way is to do testing via the logical
> replication of DDL (aka via this patch) and the other is to write an
> independent test suite as Sawada-San seems to be speculating above
> [2]. I am not sure if there is any progress yet on the independent
> test suite front yet.
Testing DDL deparsing support has been discussed before in [1], we
will also take a close look on it.
[1] https://www.postgresql.org/message-id/5477FD78.1060306%402ndquadrant.com
Regards,
Zheng
Attachments:
[application/octet-stream] 0003-Logical-replication-of-DDL-messages.patch (41.2K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/2-0003-Logical-replication-of-DDL-messages.patch)
download | inline diff:
From b193e29c362aaf379012c2fc6c2e4e648d1c0bb0 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 17:17:39 +0000
Subject: [PATCH 03/12] Logical replication of DDL messages
Integration with pgoutput:
Supports sending and receiving the DDL message using the logical
replication wire protocol. A new LogicalRepMsgType is introduced
for this purpose: LOGICAL_REP_MSG_DDLMESSAGE = 'L'.
Logical replication worker change:
Supports execution of the DDL command in the original user role and
search_path. For any new table created this way, we also set its
srsubstate in the pg_subscription_rel catalog to SUBREL_STATE_INIT,
So that DML replication can progress on this new table without
manually running "ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
TAP test:
A new TAP test 030_rep_ddl.pl is added. We mainly focused on
testing the happy path of database level replication so far.
Corner case DDLs and table level DDL replication are still to be
carefully tested.
---
.../test_decoding/expected/ddlmessages.out | 26 +-
contrib/test_decoding/sql/ddlmessages.sql | 5 +-
src/backend/replication/logical/proto.c | 63 ++++-
src/backend/replication/logical/worker.c | 264 ++++++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 82 +++++-
src/backend/tcop/utility.c | 14 +-
src/include/replication/logicalproto.h | 10 +-
src/test/subscription/t/004_sync.pl | 2 +-
src/test/subscription/t/006_rewrite.pl | 2 +-
src/test/subscription/t/008_diff_schema.pl | 2 +-
src/test/subscription/t/009_matviews.pl | 2 +-
src/test/subscription/t/012_collation.pl | 2 +-
src/test/subscription/t/013_partition.pl | 8 +-
src/test/subscription/t/030_rep_ddls.pl | 237 ++++++++++++++++
14 files changed, 684 insertions(+), 35 deletions(-)
create mode 100644 src/test/subscription/t/030_rep_ddls.pl
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 79284f9def..0376f36c24 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -2,6 +2,9 @@
SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
?column?
----------
@@ -20,23 +23,22 @@ BEGIN;
CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
-SELECT pg_drop_replication_slot('regression_slot');
-DROP TABLE test_ddlmessage;
-DROP publication mypub;
- data
---------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
(1 row)
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index 211497ee22..c23610f9b4 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -3,6 +3,10 @@ SET synchronous_commit = on;
-- turn on logical ddl message logging
CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+-- SET USER
+CREATE ROLE ddl_replication_user LOGIN SUPERUSER;
+SET SESSION AUTHORIZATION 'ddl_replication_user';
+
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
CREATE TABLE test_ddlmessage (id serial unique, data int);
@@ -20,7 +24,6 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
-\o | sed 's/role.*search_path/role: redacted, search_path/g'
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e2d2..3cfa94dd8c 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -662,6 +662,63 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
pq_sendbytes(out, message, sz);
}
+/*
+ * Read DDL MESSAGE from stream
+ */
+const char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ const char **role,
+ const char **search_path,
+ bool *transactional,
+ Size *sz)
+{
+ uint8 flags;
+ const char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *role = pq_getmsgstring(in);
+ *search_path = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* encode and send message flags */
+ if (transactional)
+ flags |= MESSAGE_TRANSACTIONAL;
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendstring(out, role);
+ pq_sendstring(out, search_path);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
/*
* Write relation description to the output stream.
*/
@@ -1218,6 +1275,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 607f719fd6..2149a9894e 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -180,6 +181,8 @@
#include "storage/proc.h"
#include "storage/procarray.h"
#include "tcop/tcopprot.h"
+#include "tcop/pquery.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -346,6 +349,10 @@ static void apply_handle_tuple_routing(ApplyExecutionData *edata,
TupleTableSlot *remoteslot,
LogicalRepTupleData *newtup,
CmdType operation);
+static void apply_execute_sql_command(const char *cmdstr,
+ const char* role,
+ const char* search_path,
+ bool isTopLevel);
/* Compute GID for two_phase transactions */
static void TwoPhaseTransactionGid(Oid subid, TransactionId xid, char *gid, int szgid);
@@ -2448,6 +2455,259 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/*
+ * Handle generic messages.
+ */
+static void
+apply_handle_ddlmessage(StringInfo s)
+{
+ XLogRecPtr lsn;
+ bool transactional;
+ Size sz;
+ const char *prefix;
+ const char *role;
+ const char *search_path;
+ const char *msg;
+
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+
+ apply_execute_sql_command(msg, role, search_path, true);
+}
+
+/*
+ * Add context to the errors produced by apply_execute_sql_command().
+ */
+static void
+execute_sql_command_error_cb(void *arg)
+{
+ errcontext("during execution of SQL statement: %s", (char *) arg);
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE stmt only */
+ char *relname = NULL; /* For CREATE TABLE stmt only */
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ /*
+ * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, CreateStmt may have been freeed up.
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ RangeVar *rv = cstmt->relation;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+ }
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * OK to analyze, rewrite, and plan this query.
+ *
+ * Switch to appropriate context for constructing query and plan trees
+ * (these can't be in the transaction context, as that will get reset
+ * when the command is COMMIT/ROLLBACK). If we have multiple
+ * parsetrees, we use a separate context for each one, so that we can
+ * free that memory before moving on to the next one. But for the
+ * last (or only) parsetree, just use MessageContext, which will be
+ * reset shortly after completion anyway. In event of an error, the
+ * per_parsetree_context will be deleted when MessageContext is reset.
+ */
+ if (lnext(parsetree_list, parsetree_item) != NULL)
+ {
+ per_parsetree_context =
+ AllocSetContextCreate(MessageContext,
+ "per-parsetree message context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+ }
+ else
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(
+ command,
+ cmdstr,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(
+ querytree_list, cmdstr, 0, NULL);
+
+ /*
+ * Done with the snapshot used for parsing/planning.
+ *
+ * While it looks promising to reuse the same snapshot for query
+ * execution (at least for simple protocol), unfortunately it causes
+ * execution to use a snapshot that has been acquired before locking
+ * any of the tables mentioned in the query. This creates user-
+ * visible anomalies, so refrain. Refer to
+ * https://postgr.es/m/flat/5075D8DF.6050500@fuzzy.cz for details.
+ */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in MessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ cmdstr,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ isTopLevel,
+ true,
+ receiver,
+ receiver,
+ NULL);
+ (*receiver->rDestroy) (receiver);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set
+ * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * new table can be replicated without having to manually run
+ * "alter subscription ... refresh publication"
+ */
+ if (commandTag == CMDTAG_CREATE_TABLE)
+ {
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+ }
+
+ /*
+ * Restore the GUC variables we set above.
+ */
+ AtEOXact_GUC(true, save_nestlevel);
+
+ /* protect against stack resets during CONCURRENTLY processing */
+ if (error_context_stack == &errcallback)
+ error_context_stack = errcallback.previous;
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
/*
* Logical replication protocol message dispatcher.
@@ -2513,6 +2773,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddlmessage(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57143..db221c7e8d 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,10 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +260,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +277,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -1656,8 +1662,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1703,70 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char * role,
+ const char *search_path, Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ TransactionId xid = InvalidTransactionId;
+ ListCell *lc;
+
+ /* Reload publications if needed before use. */
+ if (!publications_valid)
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+ if (data->publications)
+ list_free_deep(data->publications);
+
+ data->publications = LoadPublications(data->publication_names);
+ MemoryContextSwitchTo(oldctx);
+ publications_valid = true;
+ }
+
+ /* Check if ddl replication is turned on for the publications */
+ foreach(lc, data->publications)
+ {
+ Publication *pub = (Publication *) lfirst(lc);
+ /* TODO need to check relid for table level DDLs */
+ if (!pub->pubactions.pubddl_database && !pub->pubactions.pubddl_table)
+ return;
+ }
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ /*
+ * Output BEGIN if we haven't yet. Avoid for non-transactional
+ * messages.
+ */
+ if (transactional)
+ {
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+ }
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ transactional,
+ prefix,
+ role,
+ search_path,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2052,9 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl_database =
+ entry->pubactions.pubddl_table = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2112,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl_database = false;
+ entry->pubactions.pubddl_table = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2227,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl_database |= pub->pubactions.pubddl_database;
+ entry->pubactions.pubddl_table |= pub->pubactions.pubddl_table;
/*
* We want to publish the changes as the top-most ancestor
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 5fe54f742a..d0bb232af6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1268,13 +1268,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
switch (nodeTag(parsetree))
{
/*
@@ -2110,6 +2103,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8ff3..28ff562d62 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,14 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size sz, const char *message);
+extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
+ const char **role, const char **search_path,
+ bool *transactional, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/test/subscription/t/004_sync.pl b/src/test/subscription/t/004_sync.pl
index cf61fc1e0f..698c5114e6 100644
--- a/src/test/subscription/t/004_sync.pl
+++ b/src/test/subscription/t/004_sync.pl
@@ -33,7 +33,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/006_rewrite.pl b/src/test/subscription/t/006_rewrite.pl
index c924ff35f7..6c9b055eb0 100644
--- a/src/test/subscription/t/006_rewrite.pl
+++ b/src/test/subscription/t/006_rewrite.pl
@@ -23,7 +23,7 @@ $node_subscriber->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/008_diff_schema.pl b/src/test/subscription/t/008_diff_schema.pl
index 67b4026afa..2902e6fc34 100644
--- a/src/test/subscription/t/008_diff_schema.pl
+++ b/src/test/subscription/t/008_diff_schema.pl
@@ -32,7 +32,7 @@ $node_subscriber->safe_psql('postgres',
# Setup logical replication
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION tap_pub FOR ALL TABLES");
+ "CREATE PUBLICATION tap_pub FOR ALL TABLES WITH (ddl = '')");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION tap_sub CONNECTION '$publisher_connstr' PUBLICATION tap_pub"
diff --git a/src/test/subscription/t/009_matviews.pl b/src/test/subscription/t/009_matviews.pl
index 1ce696d4a4..6c586877a1 100644
--- a/src/test/subscription/t/009_matviews.pl
+++ b/src/test/subscription/t/009_matviews.pl
@@ -19,7 +19,7 @@ $node_subscriber->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION mypub FOR ALL TABLES;");
+ "CREATE PUBLICATION mypub FOR ALL TABLES WITH (ddl = '');");
$node_subscriber->safe_psql('postgres',
"CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
);
diff --git a/src/test/subscription/t/012_collation.pl b/src/test/subscription/t/012_collation.pl
index 2182f7948e..e1b5050663 100644
--- a/src/test/subscription/t/012_collation.pl
+++ b/src/test/subscription/t/012_collation.pl
@@ -76,7 +76,7 @@ $node_subscriber->safe_psql('postgres',
# set up publication, subscription
$node_publisher->safe_psql('postgres',
- q{CREATE PUBLICATION pub1 FOR ALL TABLES});
+ q{CREATE PUBLICATION pub1 FOR ALL TABLES WITH (ddl = '')});
$node_subscriber->safe_psql('postgres',
qq{CREATE SUBSCRIPTION sub1 CONNECTION '$publisher_connstr' PUBLICATION pub1 WITH (copy_data = false)}
diff --git a/src/test/subscription/t/013_partition.pl b/src/test/subscription/t/013_partition.pl
index 69f4009a14..af028e0258 100644
--- a/src/test/subscription/t/013_partition.pl
+++ b/src/test/subscription/t/013_partition.pl
@@ -25,9 +25,9 @@ $node_subscriber2->start;
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
# publisher
-$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1");
+$node_publisher->safe_psql('postgres', "CREATE PUBLICATION pub1 WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_all FOR ALL TABLES");
+ "CREATE PUBLICATION pub_all FOR ALL TABLES WITH (ddl = '')");
$node_publisher->safe_psql('postgres',
"CREATE TABLE tab1 (a int PRIMARY KEY, b text) PARTITION BY LIST (a)");
$node_publisher->safe_psql('postgres',
@@ -425,12 +425,12 @@ $node_publisher->safe_psql('postgres',
# and child tables are present but changes will be replicated via the parent's
# identity and only once.
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_viaroot FOR TABLE tab2, tab2_1, tab3_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# for tab4, we publish changes through the "middle" partitioned table
$node_publisher->safe_psql('postgres',
- "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true)"
+ "CREATE PUBLICATION pub_lower_level FOR TABLE tab4_1 WITH (publish_via_partition_root = true, ddl = '')"
);
# prepare data for the initial sync
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
new file mode 100644
index 0000000000..c88c4ea1c0
--- /dev/null
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -0,0 +1,237 @@
+
+# Copyright (c) 2022, PostgreSQL Global Development Group
+
+# Regression tests for logical replication of DDLs
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->append_conf('postgresql.conf', 'autovacuum = off');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber->start;
+
+my $node_subscriber2 = PostgreSQL::Test::Cluster->new('subscriber2');
+$node_subscriber2->init(allows_streaming => 'logical');
+$node_subscriber2->append_conf('postgresql.conf', 'autovacuum = off');
+$node_subscriber2->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1');");
+$node_subscriber->safe_psql('postgres', $ddl);
+$node_subscriber2->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+
+# mypub has pubddl_database on
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES;");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;"
+);
+# mypub2 has pubddl_database off
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub2 FOR ALL TABLES with (ddl = '');");
+$node_subscriber2->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub2 CONNECTION '$publisher_connstr' PUBLICATION mypub2;"
+);
+
+$node_publisher->wait_for_catchup('mysub');
+
+# Test simple CREATE TABLE command is replicated to subscriber
+# Test smae simple CREATE TABLE command is not replicated to subscriber2 (ddl off)
+# Test ALTER TABLE command is replicated on table test_rep
+# Test CREATE INDEX is replicated to subscriber
+# Test CREATE FUNCTION command is replicated to subscriber
+$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c3 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2);");
+$node_publisher->safe_psql('postgres', "CREATE INDEX nameindex on test_rep (name)");
+$node_publisher->safe_psql('postgres', qq{CREATE OR REPLACE FUNCTION totalRecords()
+RETURNS integer AS \$total\$
+declare
+ total integer;
+BEGIN
+ SELECT count(*) into total FROM test_rep;
+ RETURN total;
+END;
+\$total\$ LANGUAGE plpgsql;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t1");
+is($result, qq(0), 'CREATE of t1 replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't1';");
+is($result, qq(0), 'CREATE of t1 is not replicated to subscriber2');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c3 =2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(1), 'CREATE INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT totalRecords();");
+is($result, qq(2), 'CREATE of function totalRecords replicated to subscriber');
+$result = $node_subscriber2->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'CREATE FUNCTION totalrecords is not replicated to subscriber2');
+
+# Test ALTER TABLE DROP
+# Test DROP INDEX
+# Test DROP FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->safe_psql('postgres', "DROP INDEX nameindex;");
+$node_publisher->safe_psql('postgres', "DROP FUNCTION totalRecords;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_class where relname = 'nameindex'");
+is($result, qq(0), 'DROP INDEX nameindex replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_proc where proname = 'totalrecords';");
+is($result, qq(0), 'DROP FUNCTION totalrecords replicated');
+
+
+# TODO figure out how to set ON_ERROR_STOP = 0 in this test
+# Test failed CREATE/ALTER TABLE on publisher doesn't break replication
+# Table t1 already exits so expect the command to fail
+#$node_publisher->safe_psql('postgres', "CREATE TABLE t1 (a int, b varchar);");
+#$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c3;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (103, 'data103', 1013);");
+
+#$node_publisher->wait_for_catchup('mysub');
+# Verify replication still works
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+#is($result, qq(1), 'DELETE from test_rep replicated');
+
+# Test DDLs inside txn block
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE t2 (a int, b varchar);
+ALTER TABLE test_rep ADD c3 int;
+INSERT INTO test_rep VALUES (3, 'data3', 3);
+COMMIT;});
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t2;");
+is($result, qq(0), 'CREATE t2 replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep;");
+is($result, qq(2), 'ALTER test_rep ADD replicated');
+
+# Test toggling pubddl_database option off
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = '');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(f|f), 'pubddl_database turned off on mypub');
+$node_publisher->safe_psql('postgres', "CREATE TABLE t3 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't3';");
+is($result, qq(0), 'CREATE t3 is not replicated');
+
+# Test toggling pubddl_database option on
+$node_publisher->safe_psql('postgres', "ALTER PUBLICATION mypub set (ddl = 'database');");
+$result = $node_publisher->safe_psql('postgres', "SELECT pubddl_database, pubddl_table from pg_publication where pubname = 'mypub';");
+is($result, qq(t|t), 'pubddl_database turned on on mypub');
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE t4 (a int, b varchar);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 't4';");
+is($result, qq(1), 'CREATE t4 is replicated');
+
+# Test DML changes on the new table t4 are replicated
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO t4 values (2, 'b')");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from t4;");
+is($result, qq(2), 'DML Changes to t4 are replicated');
+
+# A somewhat complicated test in plpgsql block with trigger
+$node_publisher->safe_psql(
+ 'postgres', q{
+BEGIN;
+CREATE TABLE foo (a int);
+CREATE INDEX foo_idx ON foo (a);
+ALTER TABLE foo ADD COLUMN b timestamptz;
+CREATE FUNCTION foo_ts()
+RETURNS trigger AS $$
+BEGIN
+NEW.b := current_timestamp;
+RETURN NEW;
+END;
+$$
+LANGUAGE plpgsql;
+CREATE TRIGGER foo_ts BEFORE INSERT OR UPDATE ON foo
+FOR EACH ROW EXECUTE FUNCTION foo_ts();
+INSERT INTO foo VALUES (1);
+COMMIT;});
+$result = $node_publisher->safe_psql('postgres', "SELECT b from foo where a = 1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+my $result_sub = $node_subscriber->safe_psql('postgres', "SELECT b from foo where a = 1;");
+is($result, qq($result_sub), 'timestamp of insert matches');
+
+# Test CREATE SCHEMA stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE SCHEMA s1");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_namespace WHERE nspname = 's1';");
+is($result, qq(1), 'CREATE SCHEMA s1 is replicated');
+
+# Test CREATE TABLE in new schema s1 followed by insert
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t1 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO s1.t1 VALUES (2, 'b');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t1;");
+is($result, qq(2), 'CREATE TABLE s1.t1 is replicated');
+
+# Test replication works as expected with mismatched search_path on publisher and subscriber
+$node_publisher->append_conf('postgresql.conf', 'search_path = \'s1, public\'');
+$node_publisher->restart;
+# CREATE unqualified table t2, it is s1.t2 under the modified search_path
+$node_publisher->safe_psql('postgres', "CREATE TABLE t2 (a int, b varchar);");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (1, 'a');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (2, 'b');");
+$node_publisher->safe_psql('postgres', "INSERT INTO t2 VALUES (3, 'c');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.t2;");
+is($result, qq(3), 'CREATE TABLE s1.t2 is replicated');
+
+# Test owner of new table on subscriber matches the owner on publisher
+$node_publisher->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_subscriber->safe_psql('postgres', "CREATE ROLE ddl_replication_user LOGIN SUPERUSER;");
+
+$node_publisher->safe_psql('postgres', "SET SESSION AUTHORIZATION 'ddl_replication_user'; CREATE TABLE t5 (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
+is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+
+#TODO TEST certain DDLs are not replicated
+
+pass "DDL replication tests passed!";
+
+$node_subscriber->stop;
+$node_subscriber2->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.32.0
[application/octet-stream] 0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch (8.0K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/3-0005-Support-replication-of-CREATE-.-AS-.-and-SELECT-.-IN.patch)
download | inline diff:
From 4938badf12bebf461dae27cd76b6015c27fb6f53 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Thu, 24 Mar 2022 17:46:55 +0000
Subject: [PATCH 05/12] Support replication of CREATE ... AS .. and SELECT ...
INTO ... statements.
The idea is to force skipping the direct data population
(which could potentially cause data mismatch compared to the publisher)
with these command on the subscriber by force setting the skipData flag
in the intoClause of the parsetree after the logical replication worker
parses the command. The data sync will be taken care of by DML replication
after the replication of the DDL command.
---
src/backend/replication/logical/worker.c | 51 ++++++++++++++++++++----
src/backend/tcop/utility.c | 41 +++++++++++--------
src/test/subscription/t/030_rep_ddls.pl | 24 +++++++++++
3 files changed, 92 insertions(+), 24 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 2149a9894e..72711b06d6 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2545,23 +2545,58 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
Portal portal;
DestReceiver *receiver;
bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE stmt only */
- char *relname = NULL; /* For CREATE TABLE stmt only */
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
commandTag = CreateCommandTag((Node *)command);
/*
- * Remember the schemaname and relname if it's a CREATE TABLE stmt
+ * Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
- * execute the stmt. At that point, CreateStmt may have been freeed up.
+ * execute the stmt. At that point, command->stmt may have been freeed up.
*/
if (commandTag == CMDTAG_CREATE_TABLE)
{
- CreateStmt *cstmt = (CreateStmt *)command->stmt;
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
}
+ else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ RangeVar *rv = castmt->into->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ /* SELECT INTO */
+ else if (commandTag == CMDTAG_SELECT)
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ RangeVar *rv = sstmt->intoClause->rel;
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ /*
+ * Force skipping data population to avoid data inconsistency.
+ * Data should be replicated from the publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
@@ -2660,12 +2695,12 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
* Table created by DDL replication (database level) is automatically
* added to the subscription here.
*
- * Call AddSubscriptionRelState for CREATE TABEL command to set
- * the relstate to SUBREL_STATE_INIT so DML changes on this
+ * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+ * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
* new table can be replicated without having to manually run
* "alter subscription ... refresh publication"
*/
- if (commandTag == CMDTAG_CREATE_TABLE)
+ if (relname != NULL)
{
Oid relid;
Oid relnamespace = InvalidOid;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 79b5faccc0..29381b53b6 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1142,24 +1142,31 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/*
* CreateTableAsStmt can create either a table a materialized view
- * and they are handled differently.
*/
case T_CreateTableAsStmt:
{
CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+
switch(stmt->objtype)
{
+ /*
+ * Either CREATE TABLE AS or SELECT ... INTO stmt
+ * The statement is logged as is, but when we apply the
+ * CREATE TABLE AS or SELECT ... INTO statemtns on the logical
+ * replication worker, we will force the skipData flag in the
+ * intoClause. This way we avoid direct data population on the
+ * subsriber with the execution of these commands which can
+ * potentially cause data mismatch bewteen the publisher.
+ *
+ * The data sync will be handled by DML replication after the
+ * target table has been created.
+ */
case OBJECT_TABLE:
- /*
- * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
- * 1. A normal CREATE TABLE string that get's logged and replicated via
- * DDL replication.
- * 2. Insertions that get replicated by DML replication.
- */
- break;
+
+ /* CREATE MATERIALIZED VIEW */
case OBJECT_MATVIEW:
/*
- * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * Log these stmt for logical replication if
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
@@ -1304,8 +1311,17 @@ ProcessUtilitySlow(ParseState *pstate,
PG_TRY();
{
if (isCompleteQuery)
+ {
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a complete top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+ }
+
switch (nodeTag(parsetree))
{
/*
@@ -2141,13 +2157,6 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
{
- /*
- * Consider logging the DDL command if logical logging is enabled and this is
- * a complete top level query.
- */
- if (XLogLogicalInfoActive() && isTopLevel)
- LogLogicalDDLCommand(parsetree, queryString);
-
EventTriggerSQLDrop(parsetree);
EventTriggerDDLCommandEnd(parsetree);
}
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 562efe2cf7..3b15c6d9f0 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -244,6 +244,30 @@ $node_publisher->wait_for_catchup('mysub');
$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+# TEST CREATE TABLE AS stmt
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t3 AS SELECT a, b from s1.t1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t3;");
+is($result, qq(2), 'CREATE TABLE s1.t3 AS is replicated with data');
+
+# TEST CREATE TABLE AS stmt ... WITH NO DATA
+$node_publisher->safe_psql('postgres', "CREATE TABLE s1.t4 AS SELECT a, b from s1.t1 WITH NO DATA;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t4;");
+is($result, qq(0), 'CREATE TABLE s1.t4 AS is replicated with no data');
+
+# TEST SELECT INTO stmt
+$node_publisher->safe_psql('postgres', "SELECT b into s1.t6 from s1.t1 where a > 1");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
+is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0002-Support-logical-logging-and-decoding-of-DDL-command-.patch (61.4K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/4-0002-Support-logical-logging-and-decoding-of-DDL-command-.patch)
download | inline diff:
From 7ab1fd432489f3dbd056fa897786cc5315c7d8ac Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:57:23 +0000
Subject: [PATCH 02/12] Support logical logging and decoding of DDL command
string.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
A new WAL record type xl_logical_ddl_message is introduced to
support logical logging of DDL command. xl_logical_ddl_message is
similar to the existing xl_logical_message for generic message
logging. The reason for not using xl_logical_message directly as
proposed initially is I found out we need to log more information
(such as user role, search path and potentially more in the future)
than just one string, and we don’t want to make too much changes to
the existing xl_logical_message which may break its current consumers.
The logging of DDL command string is processed in function
LogLogicalDDLCommand. We categorize DDL command types into three
categories in this function:
1. replicated in database level replication only (such as CREATE
TABLE, CREATE FUNCTION).
2. replicated in database or table level replication depending on
the configuration (such as ALTER TABLE).
3. not supported for replication or pending investigation.
Support logical decoding of the new WAL record xl_logical_ddl_message.
This is similar to the logical decoding of xl_logical_message. Tests
for this change are added in the test_decoding plugin.
---
contrib/test_decoding/Makefile | 4 +-
.../test_decoding/expected/ddlmessages.out | 42 ++++
contrib/test_decoding/sql/ddlmessages.sql | 28 +++
contrib/test_decoding/test_decoding.c | 61 +++++-
src/backend/access/rmgrdesc/Makefile | 1 +
.../access/rmgrdesc/logicalddlmsgdesc.c | 54 +++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 52 +++++
src/backend/commands/tablecmds.c | 43 +++-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 99 +++++++++
src/backend/replication/logical/decode.c | 56 +++++
src/backend/replication/logical/logical.c | 91 ++++++++
.../replication/logical/reorderbuffer.c | 204 +++++++++++++++++-
src/backend/tcop/utility.c | 201 ++++++++++++++++-
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/include/access/rmgrlist.h | 1 +
src/include/commands/tablecmds.h | 3 +-
src/include/replication/ddlmessage.h | 47 ++++
src/include/replication/decode.h | 1 +
src/include/replication/output_plugin.h | 29 +++
src/include/replication/reorderbuffer.h | 39 ++++
src/test/regress/expected/publication.out | 16 +-
23 files changed, 1054 insertions(+), 21 deletions(-)
create mode 100644 contrib/test_decoding/expected/ddlmessages.out
create mode 100644 contrib/test_decoding/sql/ddlmessages.sql
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index b220906479..e58a76d3f1 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -5,7 +5,9 @@ PGFILEDESC = "test_decoding - example of a logical decoding output plugin"
REGRESS = ddl xact rewrite toast permissions decoding_in_xact \
decoding_into_rel binary prepared replorigin time messages \
- spill slot truncate stream stats twophase twophase_stream
+ spill slot truncate stream stats twophase twophase_stream \
+ ddlmessages
+
ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
new file mode 100644
index 0000000000..79284f9def
--- /dev/null
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -0,0 +1,42 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+ ?column?
+----------
+ init
+(1 row)
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: transactional: 1 prefix: role: redacted, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+(6 rows)
+
+ pg_drop_replication_slot
+--------------------------
+
+(1 row)
+
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
new file mode 100644
index 0000000000..211497ee22
--- /dev/null
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -0,0 +1,28 @@
+-- predictability
+SET synchronous_commit = on;
+-- turn on logical ddl message logging
+CREATE publication mypub FOR ALL TABLES with (ddl = 'database');
+
+SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
+
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ALTER TABLE test_ddlmessage drop c3;
+DROP TABLE test_ddlmessage;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+ROLLBACK;
+
+BEGIN;
+CREATE TABLE test_ddlmessage (id serial unique, data int);
+ALTER TABLE test_ddlmessage add c3 varchar;
+COMMIT;
+
+\o | sed 's/role.*search_path/role: redacted, search_path/g'
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+SELECT pg_drop_replication_slot('regression_slot');
+DROP TABLE test_ddlmessage;
+DROP publication mypub;
+
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index 3736da6784..a44e1f79e3 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -76,6 +76,11 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
TransactionId xid,
const char *gid);
@@ -116,6 +121,11 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
int nrelations, Relation relations[],
@@ -141,6 +151,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->filter_by_origin_cb = pg_decode_filter;
cb->shutdown_cb = pg_decode_shutdown;
cb->message_cb = pg_decode_message;
+ cb->ddlmessage_cb = pg_decode_ddlmessage;
cb->filter_prepare_cb = pg_decode_filter_prepare;
cb->begin_prepare_cb = pg_decode_begin_prepare_txn;
cb->prepare_cb = pg_decode_prepare_txn;
@@ -153,6 +164,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pg_decode_stream_commit;
cb->stream_change_cb = pg_decode_stream_change;
cb->stream_message_cb = pg_decode_stream_message;
+ cb->stream_ddlmessage_cb = pg_decode_stream_ddlmessage;
cb->stream_truncate_cb = pg_decode_stream_truncate;
}
@@ -747,7 +759,8 @@ pg_decode_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pg_decode_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
appendStringInfo(ctx->out, "message: transactional: %d prefix: %s, sz: %zu content:",
@@ -756,6 +769,19 @@ pg_decode_message(LogicalDecodingContext *ctx,
OutputPluginWrite(ctx, true);
}
+static void
+pg_decode_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+ appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ OutputPluginWrite(ctx, true);
+}
+
static void
pg_decode_stream_start(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn)
@@ -936,7 +962,8 @@ pg_decode_stream_change(LogicalDecodingContext *ctx,
static void
pg_decode_stream_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
- const char *prefix, Size sz, const char *message)
+ const char *prefix, Size sz,
+ const char *message)
{
OutputPluginPrepareWrite(ctx, true);
@@ -948,7 +975,35 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
else
{
appendStringInfo(ctx->out, "streaming message: transactional: %d prefix: %s, sz: %zu content:",
- transactional, prefix, sz);
+ transactional, prefix, sz);
+ appendBinaryStringInfo(ctx->out, message, sz);
+ }
+
+ OutputPluginWrite(ctx, true);
+}
+
+/*
+ * In streaming mode, we don't display the contents for transactional messages
+ * as the transaction can abort at a later point in time. We don't want users to
+ * see the message contents until the transaction is committed.
+ */
+static void
+pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ const char *prefix, const char * role, const char * search_path,
+ Size sz, const char *message)
+{
+ OutputPluginPrepareWrite(ctx, true);
+
+ if (transactional)
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
+ transactional, prefix, role, search_path, sz);
+ }
+ else
+ {
+ appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ transactional, prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
}
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72fd86..b8e29e8df3 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000000..7a352d540a
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *role = xlrec->message + xlrec->prefix_size;
+ char *search_path = xlrec->message + xlrec->prefix_size + xlrec->role_size;
+ char *message = xlrec->message + xlrec->prefix_size + xlrec->role_size + xlrec->search_path_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
+ xlrec->transactional ? "transactional" : "non-transactional",
+ prefix, role, search_path, xlrec->message_size);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed69244e3..6db9a593b8 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -29,6 +29,7 @@
#include "miscadmin.h"
#include "replication/decode.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "storage/standby.h"
#include "utils/builtins.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index b8ab1d8141..fad21a31d0 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1206,3 +1206,55 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(funcctx);
}
+
+/*
+ * Checks if DDL on relation (relid) need xlog for logical replication
+ */
+bool
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+{
+ List *allTablePubs = NIL;
+ List *tablePubs = NIL;
+ ListCell *lc;
+
+ /* Only replicate toplevel DDL command */
+ if (!isTopLevel)
+ return false;
+ if (relid == InvalidOid && !forAllTabPubOnly)
+ return false;
+
+ /*
+ * Log the DDL command if
+ * there is any FOR ALL TABLES publication with pubddl_database on
+ * or
+ * this TABLE belongs to any non FOR ALL publications with pubddl_table on
+ */
+ allTablePubs = GetAllTablesPublications();
+ foreach(lc, allTablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_database)
+ return true;
+ }
+
+ /*
+ * If forAllTabPubOnly is true (i.e. database level replication is required for the DDL
+ * to be logged), we can bail now since no publication has been found with pubddl_database on
+ */
+ if (forAllTabPubOnly)
+ return false;
+
+ tablePubs = GetRelationPublications(relid);
+ foreach(lc, tablePubs)
+ {
+ Oid pubid = lfirst_oid(lc);
+ Publication *pub = GetPublication(pubid);
+
+ if (pub->pubactions.pubddl_table)
+ return true;
+ }
+
+ return false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0ebacec..6d1487951f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -80,6 +80,7 @@
#include "partitioning/partbounds.h"
#include "partitioning/partdesc.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
#include "rewrite/rewriteManip.h"
@@ -1332,13 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(DropStmt *drop)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
+ bool ddlxlog = XLogLogicalInfoActive();
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1437,10 +1439,37 @@ RemoveRelations(DropStmt *drop)
/* Not there? */
if (!OidIsValid(relOid))
{
+ ddlxlog = false;
DropErrorMsgNonExistent(rel, relkind, drop->missing_ok);
continue;
}
+ /*
+ * Only log DROP RELATION cmd for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on or
+ * every relation to be dropped belongs to any non FOR ALL publications with pubddl_table on
+ */
+ if (ddlxlog)
+ {
+ Oid tableOid = InvalidOid;
+
+ if (relkind == RELKIND_RELATION)
+ tableOid = relOid;
+ else if (relkind == RELKIND_INDEX)
+ tableOid = IndexGetRelation(relOid, true);
+ /*
+ * Other relation types require database level ddl replication and are
+ * already logged in LogLogicalDDLCommand() if needed.
+ */
+ else
+ ddlxlog = false;
+
+ /* DROP RELATION or INDEX are allowed in table level DDL replication */
+ if (tableOid != InvalidOid &&
+ !ddl_need_xlog(tableOid, false, isTopLevel))
+ ddlxlog = false;
+ }
+
/*
* Decide if concurrent mode needs to be used here or not. The
* callback retrieved the rel's persistence for us.
@@ -1484,6 +1513,18 @@ RemoveRelations(DropStmt *drop)
add_exact_object_address(&obj, objects);
}
+ /* Log the Drop command for logical replication */
+ if (ddlxlog)
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
performMultipleDeletions(objects, drop->behavior, flags);
free_object_addresses(objects);
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fdeb71..f3eeb67312 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000000..f93573079a
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Simiarl to the generic logical messages, These DDL messages can be either
+ * transactional or non-transactional. Note by default DDLs in PostgreSQL are
+ * transactional.
+ * Transactional messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ * Non-transactional messages are sent to the plugin at the time when the
+ * logical decoding reads them from XLOG. This also means that transactional
+ * messages won't be delivered if the transaction was rolled back but the
+ * non-transactional one will always be delivered.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
+ size_t size, bool transactional)
+{
+ xl_logical_ddl_message xlrec;
+ const char *role;
+
+ role = GetUserNameFromId(roleoid, false);
+
+ /*
+ * Force xid to be allocated if we're emitting a transactional message.
+ */
+ if (transactional)
+ {
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+ }
+
+ xlrec.dbId = MyDatabaseId;
+ xlrec.transactional = transactional;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.role_size = strlen(role) + 1;
+ xlrec.search_path_size = strlen(namespace_search_path) + 1;
+ xlrec.message_size = size;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, role), xlrec.role_size);
+ XLogRegisterData(namespace_search_path, xlrec.search_path_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427ba73..034c7f2413 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -603,6 +604,61 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
message->message + message->prefix_size);
}
+/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ Snapshot snapshot;
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (message->transactional &&
+ !SnapBuildProcessChange(builder, xid, buf->origptr))
+ return;
+ else if (!message->transactional &&
+ (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
+ SnapBuildXactNeedsSkip(builder, buf->origptr)))
+ return;
+
+ snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
+ message->transactional,
+ message->message,
+ /* first part of message is prefix */
+ message->message + message->prefix_size,
+ /* Second part of message is role*/
+ message->message + message->prefix_size + message->role_size,
+ /* Third part of message is search_path */
+ message->message_size,
+ message->message + message->prefix_size +
+ message->role_size + message->search_path_size);
+}
+
/*
* Consolidated commit record handling between the different form of commit
* records.
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4273..3004f02433 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -73,6 +73,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +94,10 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role, const char *search_path,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +226,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +243,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +261,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1220,6 +1231,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char *search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
@@ -1535,6 +1584,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
error_context_stack = errcallback.previous;
}
+static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, const char *role,
+ const char* search_path, Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ role, search_path, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f9089c..ca01336604 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -512,6 +512,20 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.role != NULL)
+ pfree(change->data.ddlmsg.role);
+ change->data.ddlmsg.role = NULL;
+ if (change->data.ddlmsg.search_path != NULL)
+ pfree(change->data.ddlmsg.search_path);
+ change->data.ddlmsg.search_path = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -866,6 +880,64 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
}
+/*
+ * A transactional DDL message is queued to be processed upon commit and a
+ * non-transactional DDL message gets processed immediately.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
+{
+ if (transactional)
+ {
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+ }
+ else
+ {
+ ReorderBufferTXN *txn = NULL;
+ volatile Snapshot snapshot_now = snapshot;
+
+ if (xid != InvalidTransactionId)
+ txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+
+ /* setup snapshot to allow catalog access */
+ SetupHistoricSnapshot(snapshot_now, NULL);
+ PG_TRY();
+ {
+ rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+
+ TeardownHistoricSnapshot(false);
+ }
+ PG_CATCH();
+ {
+ TeardownHistoricSnapshot(true);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ }
+}
+
/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
@@ -1957,6 +2029,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message);
}
+/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn, true,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.role,
+ change->data.ddlmsg.search_path,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
@@ -2335,6 +2430,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3708,6 +3807,53 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the role including the size */
+ memcpy(data, &role_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.role,
+ role_size);
+ data += role_size;
+
+ /* write the search_path including the size */
+ memcpy(data, &search_path_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.search_path,
+ search_path_size);
+ data += search_path_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4022,6 +4168,18 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
sz += prefix_size + change->data.msg.message_size +
sizeof(Size) + sizeof(Size);
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+ Size role_size = strlen(change->data.ddlmsg.role) + 1;
+ Size search_path_size = strlen(change->data.ddlmsg.search_path) + 1;
+
+ sz += prefix_size + role_size + search_path_size +
+ change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Size) + sizeof(Size);
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
@@ -4282,8 +4440,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4297,6 +4454,49 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
change->data.msg.message_size);
data += change->data.msg.message_size;
+ break;
+ }
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+ Size role_size;
+ Size search_path_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read role */
+ memcpy(&role_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.role = MemoryContextAlloc(rb->context,
+ role_size);
+ memcpy(change->data.ddlmsg.role, data, role_size);
+ Assert(change->data.ddlmsg.role[role_size - 1] == '\0');
+ data += role_size;
+
+ /* read search_path */
+ memcpy(&search_path_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.ddlmsg.search_path = MemoryContextAlloc(rb->context,
+ search_path_size);
+ memcpy(change->data.ddlmsg.search_path, data, search_path_size);
+ Assert(change->data.ddlmsg.search_path[search_path_size - 1] == '\0');
+ data += search_path_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
break;
}
case REORDER_BUFFER_CHANGE_INVALIDATION:
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcded55..5fe54f742a 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -62,6 +62,7 @@
#include "miscadmin.h"
#include "parser/parse_utilcmd.h"
#include "postmaster/bgwriter.h"
+#include "replication/ddlmessage.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
#include "storage/fd.h"
@@ -86,7 +87,7 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -987,7 +988,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel);
}
break;
@@ -1087,6 +1088,154 @@ standard_ProcessUtility(PlannedStmt *pstmt,
CommandCounterIncrement();
}
+/*
+ * Log a DDL command for logical replication
+ * Some DDLs are only replicated in Database Level DDL replication
+ * Some can be replicated in Table Level DDL replication.
+ *
+ * Currently we focus on supporting Database Level DDL replication
+ */
+static void
+LogLogicalDDLCommand(Node *parsetree, const char *queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* Fisrtly, commands that are only supported in Database level DDL replication */
+ case T_CreateSchemaStmt:
+ case T_CreateStmt:
+ case T_CreateForeignTableStmt:
+ case T_AlterDomainStmt:
+ case T_DefineStmt:
+ case T_CompositeTypeStmt:
+ case T_CreateEnumStmt:
+ case T_CreateRangeStmt:
+ case T_AlterEnumStmt:
+ case T_ViewStmt:
+ case T_CreateFunctionStmt:
+ case T_AlterFunctionStmt:
+ case T_CreateTrigStmt:
+ case T_CreateDomainStmt:
+ case T_CreateCastStmt:
+ case T_CreateOpClassStmt:
+ case T_CreateOpFamilyStmt:
+ case T_AlterOpFamilyStmt:
+ case T_AlterOperatorStmt:
+ case T_AlterTypeStmt:
+ case T_GrantStmt:
+ case T_AlterCollationStmt:
+ /*
+ * Log these stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+
+ /*
+ * Secondly, commands that may be allowed in Table level DDL replication.
+ * These are currently handled in the later execution path of the command.
+ * Because we need to get the relation id which readily available in later
+ * code path.
+ */
+ case T_AlterTableStmt:
+ case T_IndexStmt:
+ case T_RenameStmt: /* TODO */
+ case T_AlterOwnerStmt: /* TODO */
+ break;
+
+ /* DropStmt depends on the removeType */
+ case T_DropStmt:
+ {
+ DropStmt* stmt = (DropStmt*) parsetree;
+ switch (stmt->removeType)
+ {
+ /* Maybe allowed in Table level DDL replication, handled in later code path */
+ case OBJECT_INDEX:
+ case OBJECT_TABLE:
+ break;
+ /* Drop of sequence is by logical replication of sequences separately */
+ case OBJECT_SEQUENCE:
+ break;
+ /* Drop of other objects are allowed in Database level DDL replication only */
+ case OBJECT_VIEW:
+ case OBJECT_MATVIEW:
+ case OBJECT_FOREIGN_TABLE:
+ default:
+ /*
+ * Log these DropStmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ break;
+ }
+ }
+ /*
+ * Lastly, rule out DDLs we don't replicate yet in DDL replication
+ * Some of these can be supported, we just need to investigate and run tests.
+ */
+ case T_CreateExtensionStmt:
+ case T_AlterExtensionStmt:
+ case T_AlterExtensionContentsStmt:
+ case T_CreateFdwStmt:
+ case T_AlterFdwStmt:
+ case T_CreateForeignServerStmt:
+ case T_AlterForeignServerStmt:
+ case T_CreateUserMappingStmt:
+ case T_AlterUserMappingStmt:
+ case T_DropUserMappingStmt:
+ case T_ImportForeignSchemaStmt:
+ case T_RuleStmt:
+ case T_CreateSeqStmt:
+ case T_AlterSeqStmt:
+ case T_CreateTableAsStmt:
+ case T_RefreshMatViewStmt:
+ case T_CreatePLangStmt:
+ case T_CreateConversionStmt:
+ case T_CreateTransformStmt:
+ case T_AlterTSDictionaryStmt:
+ case T_AlterTSConfigurationStmt:
+ case T_AlterTableMoveAllStmt:
+ case T_AlterObjectDependsStmt:
+ case T_AlterObjectSchemaStmt:
+ case T_CommentStmt:
+ case T_DropOwnedStmt:
+ case T_AlterDefaultPrivilegesStmt:
+ case T_CreatePolicyStmt:
+ case T_AlterPolicyStmt:
+ case T_SecLabelStmt:
+ case T_CreateAmStmt:
+ case T_CreatePublicationStmt:
+ case T_AlterPublicationStmt:
+ case T_CreateSubscriptionStmt:
+ case T_AlterSubscriptionStmt:
+ case T_DropSubscriptionStmt:
+ case T_CreateStatsStmt:
+ case T_AlterStatsStmt:
+ break;
+ default:
+ break;
+ }
+}
+
/*
* The "Slow" variant of ProcessUtility should only receive statements
* supported by the event triggers facility. Therefore, we always
@@ -1119,6 +1268,13 @@ ProcessUtilitySlow(ParseState *pstate,
if (isCompleteQuery)
EventTriggerDDLCommandStart(parsetree);
+ /*
+ * Consider logging the DDL command if logical logging is enabled and this is
+ * a top level query.
+ */
+ if (XLogLogicalInfoActive() && isTopLevel)
+ LogLogicalDDLCommand(parsetree, queryString);
+
switch (nodeTag(parsetree))
{
/*
@@ -1321,6 +1477,23 @@ ProcessUtilitySlow(ParseState *pstate,
EventTriggerAlterTableStart(parsetree);
EventTriggerAlterTableRelid(relid);
+ /*
+ * Log the ALTER TABLE command if
+ * There is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
/* ... and do it */
AlterTable(atstmt, lockmode, &atcontext);
@@ -1539,6 +1712,24 @@ ProcessUtilitySlow(ParseState *pstate,
/* ... and do it */
EventTriggerAlterTableStart(parsetree);
+
+ /*
+ * Log CREATE INDEX cmd for logical replication if
+ * there is any publication with database level ddl on or
+ * this TABLE belongs to any publication with table level ddl on.
+ */
+ if (XLogLogicalInfoActive() &&
+ ddl_need_xlog(relid, false, isTopLevel))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+
address =
DefineIndex(relid, /* OID of heap relation */
stmt,
@@ -1761,7 +1952,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt((DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -1982,7 +2173,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
{
switch (stmt->removeType)
{
@@ -1997,7 +2188,7 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(stmt);
+ RemoveRelations(pstate, stmt, isTopLevel);
break;
default:
RemoveObjects(stmt);
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17bb4c..792f438959 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721c97..9de3b8f2eb 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 5d4037f26e..68781365de 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -18,6 +18,7 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "storage/lock.h"
#include "utils/relcache.h"
@@ -27,7 +28,7 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(DropStmt *drop);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000000..1e8ef22296
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,47 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ bool transactional; /* is message transactional? */
+ Size prefix_size; /* length of prefix */
+ Size role_size; /* length of the role that executes the DDL command */
+ Size search_path_size; /* length of the search path */
+ Size message_size; /* size of the message */
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ * and null-terminated role of length role_size
+ * and null-terminated search_path of length search_path_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
+ size_t size, bool transactional);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif /* PG_LOGICAL_DDL_MESSAGE_H */
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65cf7..427a7b997d 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e697..5b1c245b72 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -88,6 +88,19 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
Size message_size,
const char *message);
+/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Filter changes by origin.
*/
@@ -199,6 +212,20 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
Size message_size,
const char *message);
+/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size message_size,
+ const char *message);
+
/*
* Callback for streaming truncates from in-progress transactions.
*/
@@ -219,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f877e5..dd89e08efc 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -56,6 +56,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +131,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ char *role;
+ char *search_path;
+ Size message_size;
+ char *message;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +441,17 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +518,18 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ bool transactional,
+ const char *prefix,
+ const char *role,
+ const char *search_path,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
+ bool transactional, const char *prefix, const char *role,
+ const char *search_path, Size message_size, const char *message);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index e9cea11a5b..5b4bee74cb 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | t | f | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
--
2.32.0
[application/octet-stream] 0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch (65.0K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/5-0001-Define-DDL-replication-levels-via-the-CREATE-PUBLICA.patch)
download | inline diff:
From 0188dbbe250a4e809edf556fad34dd37ca46a4eb Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 18 Mar 2022 16:47:16 +0000
Subject: [PATCH 01/12] Define DDL replication levels via the CREATE
PUBLICATION command, currently allow Database or Table level DDL replication.
Allow the user to configure either database level or table level
DDL replication via the CREATE PUBLICATION command as in my first
email. Two new columns are added to the pg_publication catalog to
show the DDL replication levels, test output publication.out is
updated accordingly. pg_dump is also modified to accommodate the
pg_publication catalog change.
---
src/backend/catalog/pg_publication.c | 2 +
src/backend/commands/publicationcmds.c | 89 +++++-
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/pg_dump.c | 45 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/t/002_pg_dump.pl | 10 +-
src/bin/psql/describe.c | 30 +-
src/include/catalog/pg_publication.h | 10 +
src/include/commands/defrem.h | 1 +
src/test/regress/expected/publication.out | 372 +++++++++++-----------
10 files changed, 364 insertions(+), 199 deletions(-)
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca62de..b8ab1d8141 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1006,6 +1006,8 @@ GetPublication(Oid pubid)
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
pub->pubviaroot = pubform->pubviaroot;
+ pub->pubactions.pubddl_database = pubform->pubddl_database;
+ pub->pubactions.pubddl_table = pubform->pubddl_table;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e645741e4..edceaa0c4e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -80,15 +80,18 @@ static void PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok);
static void
parse_publication_options(ParseState *pstate,
List *options,
+ bool for_all_tables,
bool *publish_given,
PublicationActions *pubactions,
bool *publish_via_partition_root_given,
- bool *publish_via_partition_root)
+ bool *publish_via_partition_root,
+ bool *ddl_level_given)
{
ListCell *lc;
*publish_given = false;
*publish_via_partition_root_given = false;
+ *ddl_level_given = false;
/* defaults */
pubactions->pubinsert = true;
@@ -96,6 +99,16 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
*publish_via_partition_root = false;
+ if (for_all_tables)
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ else
+ {
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+ }
/* Parse options */
foreach(lc, options)
@@ -154,6 +167,55 @@ parse_publication_options(ParseState *pstate,
*publish_via_partition_root_given = true;
*publish_via_partition_root = defGetBoolean(defel);
}
+ else if (strcmp(defel->defname, "ddl") == 0)
+ {
+ char *ddl_level;
+ List *ddl_list;
+ ListCell *lc;
+
+ if (*ddl_level_given)
+ errorConflictingDefElem(defel, pstate);
+
+ /*
+ * If publish option was given only the explicitly listed actions
+ * should be published.
+ */
+ pubactions->pubddl_database = false;
+ pubactions->pubddl_table = false;
+
+ *ddl_level_given = true;
+ ddl_level = defGetString(defel);
+
+ if (!SplitIdentifierString(ddl_level, ',', &ddl_list))
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("invalid list syntax for \"ddl\" option")));
+
+ /* Process the option list. */
+ foreach(lc, ddl_list)
+ {
+ char *publish_opt = (char *) lfirst(lc);
+
+ if (strcmp(publish_opt, "database") == 0)
+ {
+ if (!for_all_tables)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("DDL replication on the database level is only supported in FOR ALL TABLES publication")));
+ else
+ {
+ pubactions->pubddl_database = true;
+ pubactions->pubddl_table = true;
+ }
+ }
+ else if (strcmp(publish_opt, "table") == 0)
+ pubactions->pubddl_table = true;
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("unrecognized \"ddl\" value: \"%s\"", publish_opt)));
+ }
+ }
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -761,6 +823,7 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
Datum values[Natts_pg_publication];
HeapTuple tup;
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -803,9 +866,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
parse_publication_options(pstate,
stmt->options,
+ stmt->for_all_tables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
puboid = GetNewOidWithIndex(rel, PublicationObjectIndexId,
Anum_pg_publication_oid);
@@ -822,6 +887,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubtruncate);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
+ values[Anum_pg_publication_pubddl_database - 1] =
+ BoolGetDatum(pubactions.pubddl_database);
+ values[Anum_pg_publication_pubddl_table - 1] =
+ BoolGetDatum(pubactions.pubddl_table);
tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
@@ -908,6 +977,7 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
bool replaces[Natts_pg_publication];
Datum values[Natts_pg_publication];
bool publish_given;
+ bool ddl_level_given;
PublicationActions pubactions;
bool publish_via_partition_root_given;
bool publish_via_partition_root;
@@ -916,11 +986,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
List *root_relids = NIL;
ListCell *lc;
+ pubform = (Form_pg_publication) GETSTRUCT(tup);
+
parse_publication_options(pstate,
stmt->options,
+ pubform->puballtables,
&publish_given, &pubactions,
&publish_via_partition_root_given,
- &publish_via_partition_root);
+ &publish_via_partition_root,
+ &ddl_level_given);
pubform = (Form_pg_publication) GETSTRUCT(tup);
@@ -1024,6 +1098,15 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
replaces[Anum_pg_publication_pubtruncate - 1] = true;
}
+ if (ddl_level_given)
+ {
+ values[Anum_pg_publication_pubddl_database - 1] = BoolGetDatum(pubactions.pubddl_database);
+ replaces[Anum_pg_publication_pubddl_database - 1] = true;
+
+ values[Anum_pg_publication_pubddl_table - 1] = BoolGetDatum(pubactions.pubddl_table);
+ replaces[Anum_pg_publication_pubddl_table - 1] = true;
+ }
+
if (publish_via_partition_root_given)
{
values[Anum_pg_publication_pubviaroot - 1] = BoolGetDatum(publish_via_partition_root);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0e8fda97f8..fcc651dae5 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl_database |= pubform->pubddl_database;
+ pubdesc->pubactions.pubddl_table |= pubform->pubddl_table;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72e49..8860510ac1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3868,6 +3868,8 @@ getPublications(Archive *fout, int *numPublications)
int i_pubdelete;
int i_pubtruncate;
int i_pubviaroot;
+ int i_pubddl_database;
+ int i_pubddl_table;
int i,
ntups;
@@ -3882,23 +3884,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, p.pubddl_database, p.pubddl_table "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot, false AS pubddl_database, false AS pubddl_table "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3915,6 +3923,8 @@ getPublications(Archive *fout, int *numPublications)
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
i_pubviaroot = PQfnumber(res, "pubviaroot");
+ i_pubddl_database = PQfnumber(res, "pubddl_database");
+ i_pubddl_table = PQfnumber(res, "pubddl_table");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3939,6 +3949,10 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
+ pubinfo[i].pubddl_database =
+ (strcmp(PQgetvalue(res, i, i_pubddl_database), "t") == 0);
+ pubinfo[i].pubddl_table =
+ (strcmp(PQgetvalue(res, i, i_pubddl_table), "t") == 0);
/* Decide whether we want to dump it */
selectDumpableObject(&(pubinfo[i].dobj), fout);
@@ -4018,6 +4032,29 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
appendPQExpBufferStr(query, "'");
+ appendPQExpBufferStr(query, ", ddl = '");
+ first = true;
+
+ if (pubinfo->pubddl_database)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "database");
+ first = false;
+ }
+
+ if (pubinfo->pubddl_table)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "table");
+ first = false;
+ }
+
+ appendPQExpBufferStr(query, "'");
+
if (pubinfo->pubviaroot)
appendPQExpBufferStr(query, ", publish_via_partition_root = true");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c2906f..8c5ef01cb0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,8 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationInfo;
/*
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1f08716f69..cbfcabcfa8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2420,7 +2420,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
- \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2429,9 +2429,9 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub2
FOR ALL TABLES
- WITH (publish = \'\');',
+ WITH (publish = \'\', ddl = \'\');',
regexp => qr/^
- \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '');\E
+ \QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2440,7 +2440,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub3;',
regexp => qr/^
- \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub3 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
@@ -2449,7 +2449,7 @@ my %tests = (
create_order => 50,
create_sql => 'CREATE PUBLICATION pub4;',
regexp => qr/^
- \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate');\E
+ \QCREATE PUBLICATION pub4 WITH (publish = 'insert, update, delete, truncate', ddl = '');\E
/xm,
like => { %full_runs, section_post_data => 1, },
},
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 1a5d924a23..b07c6b1841 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,15 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_database AS \"%s\"",
+ gettext_noop("Database level DDLs"));
+ appendPQExpBuffer(&buf,
+ ",\n pubddl_table AS \"%s\"",
+ gettext_noop("Table level DDLs"));
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6181,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6198,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6212,11 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ {
+ appendPQExpBufferStr(&buf,
+ ", pubddl_database, pubddl_table");
+ }
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6265,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols += 2;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6281,11 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ {
+ printTableAddHeader(&cont, gettext_noop("Database level DDL"), true, align);
+ printTableAddHeader(&cont, gettext_noop("Table level DDL"), true, align);
+ }
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6296,11 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ {
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
+ printTableAddCell(&cont, PQgetvalue(res, i, 10), false, false);
+ }
if (!puballtables)
{
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba429..a94e152256 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -20,6 +20,7 @@
#include "catalog/genbki.h"
#include "catalog/objectaddress.h"
#include "catalog/pg_publication_d.h"
+#include "tcop/utility.h"
/* ----------------
* pg_publication definition. cpp turns this into
@@ -54,6 +55,12 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if database level ddls are published */
+ bool pubddl_database;
+
+ /* true if table level ddls are published */
+ bool pubddl_table;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +79,8 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl_database;
+ bool pubddl_table;
} PublicationActions;
typedef struct PublicationDesc
@@ -157,5 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 56d2bb6616..073f46ef1a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -15,6 +15,7 @@
#define DEFREM_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_publication.h"
#include "nodes/params.h"
#include "parser/parse_node.h"
#include "tcop/dest.h"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37dfe5..e9cea11a5b 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | t | t | t | f | f | f | t | t
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | f | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDL | Table level DDL
+--------------------------+------------+---------+---------+---------+-----------+----------+--------------------+-----------------
+ regress_publication_user | f | t | t | t | t | f | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.32.0
[application/octet-stream] 0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch (4.1K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/6-0004-Enable-replication-of-CREATE-MATERIALIZED-VIEW-AS-st.patch)
download | inline diff:
From b9b3ca9f7e5749ca0c89a88ab5b5bd990a8598a0 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 22 Mar 2022 23:47:52 +0000
Subject: [PATCH 04/12] Enable replication of CREATE MATERIALIZED VIEW AS stmt.
Add tests for both CREATE MATERIALIZED VIEW and CREATE VIEW.
---
src/backend/tcop/utility.c | 42 +++++++++++++++++++++++--
src/test/subscription/t/030_rep_ddls.pl | 18 +++++++++++
2 files changed, 58 insertions(+), 2 deletions(-)
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index d0bb232af6..79b5faccc0 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1140,6 +1140,45 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
}
break;
+ /*
+ * CreateTableAsStmt can create either a table a materialized view
+ * and they are handled differently.
+ */
+ case T_CreateTableAsStmt:
+ {
+ CreateTableAsStmt *stmt = (CreateTableAsStmt *) parsetree;
+ switch(stmt->objtype)
+ {
+ case OBJECT_TABLE:
+ /*
+ * FIXME CREATE TABLE AS stmt needs to be broken down into two parts
+ * 1. A normal CREATE TABLE string that get's logged and replicated via
+ * DDL replication.
+ * 2. Insertions that get replicated by DML replication.
+ */
+ break;
+ case OBJECT_MATVIEW:
+ /*
+ * Log CREATE MATERIALIZED VIEW AS stmt for logical replication if
+ * there is any FOR ALL TABLES publication with pubddl_database on.
+ * i.e. Database level DDL replication is on for some publication.
+ */
+ if (ddl_need_xlog(InvalidOid, true, true))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ default:
+ break;
+ }
+ break;
+ }
+
/*
* Secondly, commands that may be allowed in Table level DDL replication.
* These are currently handled in the later execution path of the command.
@@ -1155,7 +1194,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
/* DropStmt depends on the removeType */
case T_DropStmt:
{
- DropStmt* stmt = (DropStmt*) parsetree;
+ DropStmt *stmt = (DropStmt *) parsetree;
switch (stmt->removeType)
{
/* Maybe allowed in Table level DDL replication, handled in later code path */
@@ -1206,7 +1245,6 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
case T_RuleStmt:
case T_CreateSeqStmt:
case T_AlterSeqStmt:
- case T_CreateTableAsStmt:
case T_RefreshMatViewStmt:
case T_CreatePLangStmt:
case T_CreateConversionStmt:
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index c88c4ea1c0..562efe2cf7 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -226,6 +226,24 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT tableowner from pg_catalog.pg_tables where tablename = 't5';");
is($result, qq(ddl_replication_user), 'Owner of t5 is correct');
+# Test CREATE MATERIALIZED VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE MATERIALIZED VIEW s1.matview1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.matview1;");
+is($result, qq($result_sub), 'CREATE of s1.matview1 is replicated');
+
+# Test CREATE VIEW stmt is replicated
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.view1 AS SELECT a, b from s1.t1;");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.view1;");
+is($result, qq($result_sub), 'CREATE of s1.view1 is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch (4.8K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/7-0008-Fail-replication-worker-on-DDL-command-that-rewrites.patch)
download | inline diff:
From 1c240d98130bd67568b631d3c06b0877ae82e9f8 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 29 Apr 2022 16:57:03 +0000
Subject: [PATCH 08/12] Fail replication worker on DDL command that rewrites
table using volatile functions, such as ALTER TABLE tab ADD COLUMN col
DEFAULT volatile_expr. This is to avoid data mismatch compared to the
publisher. We can potentially unblock this type of command when table rewrite
is supported in logical replication.
---
src/backend/replication/logical/worker.c | 44 ++++++++++++++++++++++++
src/test/subscription/t/030_rep_ddls.pl | 22 ++++++++++++
2 files changed, 66 insertions(+)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 72711b06d6..39702f4a0f 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -157,6 +157,7 @@
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
#include "parser/analyze.h"
+#include "parser/parse_expr.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -2550,6 +2551,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
commandTag = CreateCommandTag((Node *)command);
+ /* The following DDL commands need special handling */
+
/*
* Remember the schemaname and relname if the cmd is going to create a table
* because we will need them for some post-processing after we
@@ -2597,6 +2600,47 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
sstmt->intoClause->skipData = true;
}
}
+ /*
+ * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
+ * Until we support logical replication of table rewrite, see ATRewriteTables()
+ * for details on table rewrite.
+ */
+ else if (commandTag == CMDTAG_ALTER_TABLE)
+ {
+ AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
+ ListCell *lc;
+
+ foreach(lc, atstmt->cmds)
+ {
+ AlterTableCmd *cmd = lfirst_node(AlterTableCmd, lc);
+
+ if (cmd->subtype == AT_AddColumn)
+ {
+ ColumnDef *colDef;
+ ListCell *c;
+
+ colDef = castNode(ColumnDef, cmd->def);
+ foreach(c, colDef->constraints)
+ {
+ Constraint *con = lfirst_node(Constraint, c);
+
+ if (con->contype == CONSTR_DEFAULT)
+ {
+ Node *expr;
+ ParseState *pstate = make_parsestate(NULL);
+
+ expr = transformExpr(pstate, copyObject(con->raw_expr), EXPR_KIND_COLUMN_DEFAULT);
+ if (contain_volatile_functions(expr))
+ {
+ elog(ERROR,
+ "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
+ cmdstr);
+ }
+ }
+ }
+ }
+ }
+ }
/*
* Set up a snapshot if parse analysis/planning will need one.
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 34b9d51eb1..b4df1bfefd 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -386,7 +386,29 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
is($result, qq(1), 'DDLs in procedure are replicated');
+# Test Alter table alter column type stmt is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN name type TEXT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT data_type FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'name';");
+is($result, qq(text), 'Alter table column type stmt is replicated');
+
+# Test Alter table add column default 0.01 is replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN non_volatile double precision DEFAULT 0.01;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
+is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+
#TODO TEST certain DDLs are not replicated
+# Test DDL statement that rewrites table with volatile functions are not replicated
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'volatile';");
+is($result, qq(1), 'Alter table add column default random() is executed on the publisher DB.');
+
+$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch (8.3K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/8-0009-Support-replication-of-DDL-type-T_RenameStmt-table-r.patch)
download | inline diff:
From ac6778fa3348db8522c3177839659a2c38484497 Mon Sep 17 00:00:00 2001
From: PonyboyYbr <94borelyang@gmail.com>
Date: Wed, 11 May 2022 15:46:40 -0700
Subject: [PATCH 09/12] Support replication of DDL type T_RenameStmt: table
rename is allowed in both database level and table level DDL replication.
Rename of other objects are only allowed in database level DDL replication.
Co-authored-by: Borui Yang <boruiyan@amazon.com>
---
src/backend/commands/alter.c | 5 ++--
src/backend/commands/tablecmds.c | 15 ++++++++++-
src/backend/tcop/utility.c | 20 ++++++++++++---
src/include/commands/alter.h | 3 ++-
src/include/commands/tablecmds.h | 2 +-
src/test/subscription/t/030_rep_ddls.pl | 33 +++++++++++++++++++++++--
6 files changed, 67 insertions(+), 11 deletions(-)
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 5456b8222b..1ca8f99e0c 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -328,7 +328,8 @@ AlterObjectRename_internal(Relation rel, Oid objectId, const char *new_name)
* Return value is the address of the renamed object.
*/
ObjectAddress
-ExecRenameStmt(RenameStmt *stmt)
+ExecRenameStmt(ParseState *pstate, RenameStmt *stmt,
+ bool isCompleteQuery)
{
switch (stmt->renameType)
{
@@ -354,7 +355,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_MATVIEW:
case OBJECT_INDEX:
case OBJECT_FOREIGN_TABLE:
- return RenameRelation(stmt);
+ return RenameRelation(pstate, stmt, isCompleteQuery);
case OBJECT_COLUMN:
case OBJECT_ATTRIBUTE:
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3c9b6409ca..65386f2641 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3820,11 +3820,12 @@ RenameConstraint(RenameStmt *stmt)
* RENAME
*/
ObjectAddress
-RenameRelation(RenameStmt *stmt)
+RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
{
bool is_index_stmt = stmt->renameType == OBJECT_INDEX;
Oid relid;
ObjectAddress address;
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/*
* Grab an exclusive lock on the target table, index, sequence, view,
@@ -3872,6 +3873,18 @@ RenameRelation(RenameStmt *stmt)
is_index_stmt = obj_is_index;
}
+ if (ddlxlog &&
+ ddl_need_xlog(relid, false))
+ {
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ pstate->p_sourcetext,
+ strlen(pstate->p_sourcetext),
+ transactional);
+ }
+
/* Do the work */
RenameRelationInternal(relid, stmt->newname, false, is_index_stmt);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index e9e7567209..1abd77a60e 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1002,7 +1002,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecRenameStmt(stmt);
+ ExecRenameStmt(pstate, stmt, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1195,8 +1195,20 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
case T_AlterTableStmt:
case T_IndexStmt:
- case T_RenameStmt: /* TODO */
- case T_AlterOwnerStmt: /* TODO */
+ case T_RenameStmt:
+ {
+ RenameStmt *stmt = (RenameStmt *) parsetree;
+ if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
+ bool transactional = true;
+ const char* prefix = "";
+ LogLogicalDDLMessage(prefix,
+ GetUserId(),
+ queryString,
+ strlen(queryString),
+ transactional);
+ }
+ }
+ case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
break;
/* DropStmt depends on the removeType */
@@ -2008,7 +2020,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_RenameStmt:
- address = ExecRenameStmt((RenameStmt *) parsetree);
+ address = ExecRenameStmt(pstate, (RenameStmt *) parsetree, isCompleteQuery);
break;
case T_AlterObjectDependsStmt:
diff --git a/src/include/commands/alter.h b/src/include/commands/alter.h
index 52f5e6f6d2..df9333eb93 100644
--- a/src/include/commands/alter.h
+++ b/src/include/commands/alter.h
@@ -17,9 +17,10 @@
#include "catalog/dependency.h"
#include "catalog/objectaddress.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_node.h"
#include "utils/relcache.h"
-extern ObjectAddress ExecRenameStmt(RenameStmt *stmt);
+extern ObjectAddress ExecRenameStmt(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern ObjectAddress ExecAlterObjectDependsStmt(AlterObjectDependsStmt *stmt,
ObjectAddress *refAddress);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 24106de2e5..f96bb57d56 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -74,7 +74,7 @@ extern ObjectAddress renameatt(RenameStmt *stmt);
extern ObjectAddress RenameConstraint(RenameStmt *stmt);
-extern ObjectAddress RenameRelation(RenameStmt *stmt);
+extern ObjectAddress RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery);
extern void RenameRelationInternal(Oid myrelid,
const char *newrelname, bool is_internal,
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index b4df1bfefd..51126b489b 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -30,7 +30,6 @@ $node_subscriber->safe_psql('postgres', $ddl);
$node_subscriber2->safe_psql('postgres', $ddl);
my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
-
# mypub has pubddl_database on
$node_publisher->safe_psql('postgres',
"CREATE PUBLICATION mypub FOR ALL TABLES;");
@@ -269,7 +268,7 @@ $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;")
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
# TEST Create DomainStmt
-$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\\s');");
$node_publisher->wait_for_catchup('mysub');
@@ -348,6 +347,36 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST RenameStmt for FUNCTION
+$node_publisher->safe_psql('postgres', "ALTER FUNCTION add RENAME TO plus;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='plus';");
+is($result, qq(1), 'RENAME FUNCTION Stmt is replicated');
+
+#TEST RenameStmt for table
+$node_publisher->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_publisher->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_publisher->safe_psql('db1',
+ "CREATE PUBLICATION mypub3 FOR TABLE t7 with (ddl = 'table');");
+my $publisher_connstr_db1 = $node_publisher->connstr . ' dbname=db1';
+$node_subscriber->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t7 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1', "CREATE TABLE t8 (id int primary key, name varchar);");
+$node_subscriber->safe_psql('db1',
+ "CREATE SUBSCRIPTION mysub3 CONNECTION '$publisher_connstr_db1' PUBLICATION mypub3;"
+);
+$node_publisher->wait_for_catchup('mysub3');
+$node_publisher->safe_psql('db1', "ALTER TABLE t7 RENAME TO newt7;");
+$node_publisher->safe_psql('db1', "ALTER TABLE t8 RENAME TO newt8;");
+$node_publisher->wait_for_catchup('mysub3');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt7';");
+is($result, qq(1), 'Rename t7 to newt7 is replicated');
+$result = $node_subscriber->safe_psql('db1', "SELECT count(*) from pg_tables where tablename = 'newt8';");
+is($result, qq(0), 'Rename t8 to newt8 is not replicated');
+
#TEST DDL in function
$node_publisher->safe_psql('postgres', qq{
CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
--
2.32.0
[application/octet-stream] 0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch (17.5K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/9-0007-Enable-logging-and-replication-of-DDLs-executed-insi.patch)
download | inline diff:
From 22ebcce07b52dcfde005564932273b21f1d4baf8 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 15 Apr 2022 02:02:10 +0000
Subject: [PATCH 07/12] Enable logging and replication of DDLs executed inside
function and procedures. Also fixed a bug where DDL with SUBCOMMAND gets
logged twice.
---
.../test_decoding/expected/ddlmessages.out | 47 +++++++++++++++++--
contrib/test_decoding/sql/ddlmessages.sql | 19 +++++++-
src/backend/catalog/pg_publication.c | 5 +-
src/backend/commands/tablecmds.c | 6 +--
src/backend/tcop/utility.c | 26 +++++-----
src/include/catalog/pg_publication.h | 2 +-
src/include/commands/tablecmds.h | 3 +-
src/test/subscription/t/030_rep_ddls.pl | 38 +++++++++++++++
8 files changed, 121 insertions(+), 25 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 0376f36c24..823029d03d 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -11,7 +11,7 @@ SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_d
init
(1 row)
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -24,9 +24,9 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ data
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
@@ -34,6 +34,44 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+SELECT func_ddl ('tab_from_func');
+ func_ddl
+----------
+
+(1 row)
+
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+ data
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
+ LANGUAGE plpgsql;
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ BEGIN
+ table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
+ table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
+ COMMIT
+(7 rows)
+
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
--------------------------
@@ -41,4 +79,5 @@ SELECT pg_drop_replication_slot('regression_slot');
(1 row)
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/contrib/test_decoding/sql/ddlmessages.sql b/contrib/test_decoding/sql/ddlmessages.sql
index c23610f9b4..3082671488 100644
--- a/contrib/test_decoding/sql/ddlmessages.sql
+++ b/contrib/test_decoding/sql/ddlmessages.sql
@@ -9,7 +9,7 @@ SET SESSION AUTHORIZATION 'ddl_replication_user';
SELECT 'init' FROM pg_create_logical_replication_slot('regression_slot', 'test_decoding');
-CREATE TABLE test_ddlmessage (id serial unique, data int);
+CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
ALTER TABLE test_ddlmessage drop c3;
DROP TABLE test_ddlmessage;
@@ -25,7 +25,24 @@ ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
+-- Test logging DDL in function
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS $$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+$$
+LANGUAGE plpgsql;
+
+SELECT func_ddl ('tab_from_func');
+SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
+
SELECT pg_drop_replication_slot('regression_slot');
DROP TABLE test_ddlmessage;
+DROP TABLE tab_from_func;
DROP publication mypub;
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index fad21a31d0..d5e4f3713d 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1211,15 +1211,12 @@ pg_get_publication_tables(PG_FUNCTION_ARGS)
* Checks if DDL on relation (relid) need xlog for logical replication
*/
bool
-ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel)
+ddl_need_xlog(Oid relid, bool forAllTabPubOnly)
{
List *allTablePubs = NIL;
List *tablePubs = NIL;
ListCell *lc;
- /* Only replicate toplevel DDL command */
- if (!isTopLevel)
- return false;
if (relid == InvalidOid && !forAllTabPubOnly)
return false;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6d1487951f..3c9b6409ca 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1333,14 +1333,14 @@ DropErrorMsgWrongType(const char *relname, char wrongkind, char rightkind)
* DROP MATERIALIZED VIEW, DROP FOREIGN TABLE
*/
void
-RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
+RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
{
ObjectAddresses *objects;
char relkind;
ListCell *cell;
int flags = 0;
LOCKMODE lockmode = AccessExclusiveLock;
- bool ddlxlog = XLogLogicalInfoActive();
+ bool ddlxlog = XLogLogicalInfoActive() && isCompleteQuery;
/* DROP CONCURRENTLY uses a weaker lock, and has some restrictions */
if (drop->concurrent)
@@ -1466,7 +1466,7 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel)
/* DROP RELATION or INDEX are allowed in table level DDL replication */
if (tableOid != InvalidOid &&
- !ddl_need_xlog(tableOid, false, isTopLevel))
+ !ddl_need_xlog(tableOid, false))
ddlxlog = false;
}
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 29381b53b6..e9e7567209 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -87,7 +87,8 @@ static void ProcessUtilitySlow(ParseState *pstate,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc);
-static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel);
+static void ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery);
/*
* CommandIsReadOnly: is an executable query read-only?
@@ -988,7 +989,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
context, params, queryEnv,
dest, qc);
else
- ExecDropStmt(pstate, stmt, isTopLevel);
+ ExecDropStmt(pstate, stmt, isTopLevel, context != PROCESS_UTILITY_SUBCOMMAND);
}
break;
@@ -1128,7 +1129,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1170,7 +1171,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1221,7 +1222,7 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
* there is any FOR ALL TABLES publication with pubddl_database on.
* i.e. Database level DDL replication is on for some publication.
*/
- if (ddl_need_xlog(InvalidOid, true, true))
+ if (ddl_need_xlog(InvalidOid, true))
{
bool transactional = true;
const char* prefix = "";
@@ -1318,7 +1319,7 @@ ProcessUtilitySlow(ParseState *pstate,
* Consider logging the DDL command if logical logging is enabled and this is
* a complete top level query.
*/
- if (XLogLogicalInfoActive() && isTopLevel)
+ if (XLogLogicalInfoActive())
LogLogicalDDLCommand(parsetree, queryString);
}
@@ -1530,7 +1531,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1766,7 +1768,8 @@ ProcessUtilitySlow(ParseState *pstate,
* this TABLE belongs to any publication with table level ddl on.
*/
if (XLogLogicalInfoActive() &&
- ddl_need_xlog(relid, false, isTopLevel))
+ isCompleteQuery &&
+ ddl_need_xlog(relid, false))
{
bool transactional = true;
const char* prefix = "";
@@ -1999,7 +2002,7 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_DropStmt:
- ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel);
+ ExecDropStmt(pstate, (DropStmt *) parsetree, isTopLevel, isCompleteQuery);
/* no commands stashed for DROP */
commandCollected = true;
break;
@@ -2220,7 +2223,8 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
* Dispatch function for DropStmt
*/
static void
-ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
+ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel,
+ bool isCompleteQuery)
{
switch (stmt->removeType)
{
@@ -2235,7 +2239,7 @@ ExecDropStmt(ParseState *pstate, DropStmt *stmt, bool isTopLevel)
case OBJECT_VIEW:
case OBJECT_MATVIEW:
case OBJECT_FOREIGN_TABLE:
- RemoveRelations(pstate, stmt, isTopLevel);
+ RemoveRelations(pstate, stmt, isCompleteQuery);
break;
default:
RemoveObjects(stmt);
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index a94e152256..8c114b2447 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -166,6 +166,6 @@ extern Bitmapset *pub_collist_to_bitmapset(Bitmapset *columns, Datum pubcols,
extern Oid get_publication_oid(const char *pubname, bool missing_ok);
extern char *get_publication_name(Oid pubid, bool missing_ok);
-extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly, bool isTopLevel);
+extern bool ddl_need_xlog(Oid relid, bool forAllTabPubOnly);
#endif /* PG_PUBLICATION_H */
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 68781365de..24106de2e5 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -28,7 +28,8 @@ struct AlterTableUtilityContext; /* avoid including tcop/utility.h here */
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString);
-extern void RemoveRelations(ParseState *pstate, DropStmt *drop, bool isTopLevel);
+extern void RemoveRelations(ParseState *pstate, DropStmt *drop,
+ bool isCompleteQuery);
extern Oid AlterTableLookupRelation(AlterTableStmt *stmt, LOCKMODE lockmode);
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 4e05212068..34b9d51eb1 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -348,6 +348,44 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
is($result, qq(1), 'CreateCast Stmt is replicated');
+#TEST DDL in function
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20))
+RETURNS VOID AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END;
+\$\$
+LANGUAGE plpgsql;});
+
+$node_publisher->safe_psql('postgres', "SELECT func_ddl('func_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.func_table where c3 = 22;");
+is($result, qq(1), 'DDLs in function are replicated');
+
+#TEST DDL in procedure
+$node_publisher->safe_psql('postgres', qq{
+CREATE OR REPLACE procedure proc_ddl (tname varchar(20))
+LANGUAGE plpgsql AS \$\$
+BEGIN
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname);
+ execute format('ALTER TABLE %I ADD c3 int', tname);
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname);
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname);
+END \$\$;});
+
+$node_publisher->safe_psql('postgres', "CALL proc_ddl('proc_table');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM s1.proc_table where c3 = 22;");
+is($result, qq(1), 'DDLs in procedure are replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0010-Handle-partitioned-table-creation-on-the-apply-worke.patch (11.5K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/10-0010-Handle-partitioned-table-creation-on-the-apply-worke.patch)
download | inline diff:
From 19a3952e7db725a41d726bbfe54bbda6bf401975 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 13 May 2022 19:47:44 +0000
Subject: [PATCH 10/12] Handle partitioned table creation on the apply worker:
whether a partitioned table should be added to pg_subscription_rel catalog
depends on the setting of publish_via_partition_root of the publication. Thus
we need to connect to the source DB and check whehter the partitioned table
should be subscribed.
---
src/backend/commands/subscriptioncmds.c | 54 +++++++++++++++++++++-
src/backend/replication/logical/worker.c | 35 ++++++++++++---
src/include/commands/subscriptioncmds.h | 3 ++
src/test/regress/expected/psql.out | 6 +--
src/test/subscription/t/030_rep_ddls.pl | 57 ++++++++++++++++++++++++
5 files changed, 145 insertions(+), 10 deletions(-)
diff --git a/src/backend/commands/subscriptioncmds.c b/src/backend/commands/subscriptioncmds.c
index 83e6eae855..25d0a23db9 100644
--- a/src/backend/commands/subscriptioncmds.c
+++ b/src/backend/commands/subscriptioncmds.c
@@ -23,7 +23,6 @@
#include "catalog/namespace.h"
#include "catalog/objectaccess.h"
#include "catalog/objectaddress.h"
-#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_type.h"
#include "commands/defrem.h"
@@ -746,6 +745,59 @@ CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
return myself;
}
+/*
+ * Check if a partitioned table is being published by any publication subscirbed by the subscription.
+ * Whether a partitioned table is published also depends on the publication option
+ * publish_via_partition_root. But the subscriber doesn't know the setting of publish_via_partition_root,
+ * this is why we need to check the source DB so that we can decide whether to subscribe to the partitioned
+ * table (could be either root or leaf table) during replication of create partitioned table.
+ */
+bool
+IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name)
+{
+ char *err;
+ WalRcvExecResult *res;
+ StringInfoData cmd;
+ Oid tableRow[1] = {BOOLOID};
+ bool result;
+
+ WalReceiverConn *wrconn;
+ /* Load the library providing us libpq calls. */
+ load_file("libpqwalreceiver", false);
+
+ /* Try to connect to the publisher. */
+ wrconn = walrcv_connect(sub->conninfo, true, sub->name, &err);
+ if (!wrconn)
+ ereport(ERROR,
+ (errcode(ERRCODE_CONNECTION_FAILURE),
+ errmsg("could not connect to the publisher: %s", err)));
+
+ PG_TRY();
+ {
+ initStringInfo(&cmd);
+ appendStringInfoString(&cmd, "SELECT TRUE\n"
+ " FROM pg_catalog.pg_publication_tables t\n"
+ " WHERE t.pubname IN (");
+ get_publications_str(sub->publications, &cmd, true);
+ appendStringInfoString(&cmd, ") AND t.schemaname = '");
+ appendStringInfoString(&cmd, schema_name);
+ appendStringInfoString(&cmd, "' AND t.tablename = '");
+ appendStringInfoString(&cmd, table_name);
+ appendStringInfoString(&cmd, "'");
+
+ res = walrcv_exec(wrconn, cmd.data, 1, tableRow);
+ pfree(cmd.data);
+ result = tuplestore_tuple_count(res->tuplestore) > 0;
+ }
+ PG_FINALLY();
+ {
+ walrcv_disconnect(wrconn);
+ }
+ PG_END_TRY();
+
+ return result;
+}
+
static void
AlterSubscription_refresh(Subscription *sub, bool copy_data,
List *validate_publications)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 39702f4a0f..1108030179 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -143,6 +143,7 @@
#include "catalog/pg_subscription.h"
#include "catalog/pg_subscription_rel.h"
#include "catalog/pg_tablespace.h"
+#include "commands/subscriptioncmds.h"
#include "commands/tablecmds.h"
#include "commands/tablespace.h"
#include "commands/trigger.h"
@@ -2548,6 +2549,7 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
bool snapshot_set = false;
char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ bool is_partitioned_table = false;
commandTag = CreateCommandTag((Node *)command);
@@ -2564,6 +2566,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
RangeVar *rv = cstmt->relation;
schemaname = rv->schemaname;
relname = rv->relname;
+ if (cstmt->inhRelations != NIL || cstmt->partspec != NULL)
+ is_partitioned_table = true;
}
else if (commandTag == CMDTAG_CREATE_TABLE_AS)
{
@@ -2765,12 +2769,31 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
if (relid != InvalidOid)
{
- AddSubscriptionRelState(MySubscription->oid, relid,
- SUBREL_STATE_INIT,
- InvalidXLogRecPtr);
- ereport(DEBUG1,
- (errmsg_internal("table \"%s\" added to subscription \"%s\"",
- relname, MySubscription->name)));
+ bool subscribe_table = true;
+
+ if (is_partitioned_table)
+ {
+ Relation rel = RelationIdGetRelation(relid);
+ char *table_name = RelationGetRelationName(rel);
+ char *schema_name = get_namespace_name(RelationGetNamespace(rel));
+ /*
+ * Connect to the source DB and check whehter the partitioned table should be subscribed.
+ * Because it depends on the setting of publish_via_partition_root, which the subscription
+ * doesn't know.
+ */
+ subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
+ RelationClose(rel);
+ }
+
+ if (subscribe_table)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
}
}
}
diff --git a/src/include/commands/subscriptioncmds.h b/src/include/commands/subscriptioncmds.h
index 2cbe7d7b65..451953af60 100644
--- a/src/include/commands/subscriptioncmds.h
+++ b/src/include/commands/subscriptioncmds.h
@@ -16,6 +16,7 @@
#define SUBSCRIPTIONCMDS_H
#include "catalog/objectaddress.h"
+#include "catalog/pg_subscription.h"
#include "parser/parse_node.h"
extern ObjectAddress CreateSubscription(ParseState *pstate, CreateSubscriptionStmt *stmt,
@@ -26,4 +27,6 @@ extern void DropSubscription(DropSubscriptionStmt *stmt, bool isTopLevel);
extern ObjectAddress AlterSubscriptionOwner(const char *name, Oid newOwnerId);
extern void AlterSubscriptionOwner_oid(Oid subid, Oid newOwnerId);
+extern bool IsPartitionedTablePublishedOnSource(Subscription *sub, char *schema_name, char *table_name);
+
#endif /* SUBSCRIPTIONCMDS_H */
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1241..3429e34339 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | Database level DDLs | Table level DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+---------------------+------------------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 51126b489b..a03d598266 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -431,6 +431,63 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep where non_volatile = 0.01;");
is($result, qq(2), 'Alter table add column default 0.01 is replicated');
+# Test partitioned table creation is replicated based on the setting of publish_via_partition_root
+$node_publisher->safe_psql('postgres', qq(
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+$result = $node_publisher->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table is created and populated');
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.test_part_a;");
+is($result, qq(2), 'Partitioned table and data is replicated');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(0), 'Root table of the partitioned table is not subscribed since publish_via_partition_root is false by default');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(2), 'Only leaf tables of the partitioned table are subscribed since publish_via_partition_root is false by default');
+
+# Test only root partition table is subscribed when publish_via_partition_root is enabled
+$node_publisher->safe_psql('postgres', qq(
+ DROP TABLE s1.test_part_a;
+ ALTER PUBLICATION mypub SET (publish_via_partition_root = true);
+ CREATE TABLE s1.test_part_a (a int, b int, c int) PARTITION BY LIST (a);
+
+ CREATE TABLE s1.test_part_a_1 PARTITION OF s1.test_part_a FOR VALUES IN (1,2,3,4,5);
+ ALTER TABLE s1.test_part_a_1 ADD PRIMARY KEY (a);
+ ALTER TABLE s1.test_part_a_1 REPLICA IDENTITY USING INDEX test_part_a_1_pkey;
+
+ CREATE TABLE s1.test_part_a_2 PARTITION OF s1.test_part_a FOR VALUES IN (6,7,8,9,10);
+ ALTER TABLE s1.test_part_a_2 ADD PRIMARY KEY (b);
+ ALTER TABLE s1.test_part_a_2 REPLICA IDENTITY USING INDEX test_part_a_2_pkey;
+
+ -- initial data, one row in each partition
+ INSERT INTO s1.test_part_a VALUES (1, 3);
+ INSERT INTO s1.test_part_a VALUES (6, 4);
+));
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a'::regclass::oid;");
+is($result, qq(1), 'Only root table of the partitioned table is subscribed since publish_via_partition_root is enabled');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
+is($result, qq(0), 'Leaf tables of the partitioned table are not subscribed since publish_via_partition_root is enabled');
+
#TODO TEST certain DDLs are not replicated
# Test DDL statement that rewrites table with volatile functions are not replicated
$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
--
2.32.0
[application/octet-stream] 0006-Add-couple-tests-for-DDL-replication-case.patch (4.5K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/11-0006-Add-couple-tests-for-DDL-replication-case.patch)
download | inline diff:
From 85f9568fb52490e8cef0f3fd4b7f5b2df0760aa3 Mon Sep 17 00:00:00 2001
From: Borui Yang <boruiyan@amazon.com>
Date: Mon, 11 Apr 2022 18:02:21 +0000
Subject: [PATCH 06/12] Add couple tests for DDL replication case
---
src/test/subscription/t/030_rep_ddls.pl | 80 +++++++++++++++++++++++++
1 file changed, 80 insertions(+)
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index 3b15c6d9f0..4e05212068 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -268,6 +268,86 @@ $node_publisher->wait_for_catchup('mysub');
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from s1.t6;");
is($result, qq(1), 'SELECT INTO s1.t6 is replicated with data');
+# TEST Create DomainStmt
+$node_publisher->safe_psql('postgres', "CREATE DOMAIN s1.space_check AS VARCHAR NOT NULL CHECK (value !~ '\s');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(t), 'CreateDomain Stmt is replicatted');
+
+# TEST AlterDomainStmt
+$node_publisher->safe_psql('postgres', "Alter domain s1.space_check drop not null;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT t.typnotnull from pg_catalog.pg_type t where t.typname='space_check';");
+is($result, qq(f), 'ALTER DOMAIN Stmt is replicated');
+
+#TEST DEFINE Stmt
+$node_publisher->safe_psql('postgres', "CREATE AGGREGATE s1.inc_sum(int) (sfunc = int4pl,stype = int,initcond = 10);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='inc_sum';");
+is($result, qq(1), 'Define stmt is replicated');
+
+#TEST CompositeTypeStmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.compfoo AS (f1 int, f2 text);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='compfoo';");
+is($result, qq(1), 'CompositeType Stmt is replicated');
+
+#TEST CreateEnum Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE s1.mood AS ENUM ('sad', 'ok', 'happy');");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='mood';");
+is($result, qq(1), 'CreateEnumType Stmt is replicated');
+
+#TEST AlterEnum Stmt
+$node_publisher->safe_psql('postgres', "ALTER TYPE s1.mood RENAME VALUE 'sad' to 'fine';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_enum e, pg_catalog.pg_type t WHERE e.enumtypid = t.oid AND t.typname='mood' AND e.enumlabel='fine';");
+is($result, qq(1), 'AlterEnumType Stmt is replicated');
+
+#TEST CreateRange Stmt
+$node_publisher->safe_psql('postgres', "CREATE TYPE floatrange AS RANGE (subtype = float8,subtype_diff = float8mi);");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_type t where t.typname='floatrange';");
+is($result, qq(1), 'CreateRange Stmt is replicated');
+
+#TEST VIEW Stmt
+$node_publisher->safe_psql('postgres', "CREATE VIEW s1.vista AS SELECT 'Hello World';");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_class c where c.relname='vista';");
+is($result, qq(1), 'VIEW Stmt is replicated');
+
+#TEST CreateFunction Stmt
+$node_publisher->safe_psql('postgres', "CREATE FUNCTION s1.add(a integer, b integer) RETURNS integer LANGUAGE SQL IMMUTABLE RETURN a + b;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_catalog.pg_proc p where p.proname='add';");
+is($result, qq(1), 'CreateFunction Stmt is replicated');
+
+#TEST CreateCast Stmt
+$node_publisher->safe_psql('postgres', "CREATE CAST (int AS int4) WITH FUNCTION add(int,int) AS ASSIGNMENT;");
+
+$node_publisher->wait_for_catchup('mysub');
+
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_catalog.pg_cast c, pg_catalog.pg_proc p WHERE p.proname='add' AND c.castfunc=p.oid;");
+is($result, qq(1), 'CreateCast Stmt is replicated');
+
#TODO TEST certain DDLs are not replicated
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch (7.3K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/12-0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch)
download | inline diff:
From 7a94899f8d92b5878d9551e7c0b3eb620c9e8a4b Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Thu, 16 Jun 2022 05:25:40 +0000
Subject: [PATCH 12/12] Support replication of ALTER TABLE commands that
rewrite table with potentially volatile functions. This is done by enabling
logical replication of table rewrite and converting the rewrite inserts to
updates which can be replayed on the subscriber without violating primary key
constraint.
An improvement is to only replicate the rewrite insert/updates when
a volatile function is used to generate the rewritten value.
---
src/backend/replication/logical/worker.c | 41 -------------------
src/backend/replication/pgoutput/pgoutput.c | 45 +++++++++++++++++++--
src/test/subscription/t/030_rep_ddls.pl | 14 ++++---
3 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index ef79d10115..698ae14270 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2548,47 +2548,6 @@ preprocess_ddl(RawStmt *command, char **schemaname, char **relname, bool *is_par
}
break;
}
- /*
- * ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
- * Until we support logical replication of table rewrite, see ATRewriteTables()
- * for details on table rewrite.
- */
- case T_AlterTableStmt:
- {
- AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
- ListCell *lc;
-
- foreach(lc, atstmt->cmds)
- {
- AlterTableCmd *cmd = lfirst_node(AlterTableCmd, lc);
-
- if (cmd->subtype == AT_AddColumn)
- {
- ColumnDef *colDef;
- ListCell *c;
-
- colDef = castNode(ColumnDef, cmd->def);
- foreach(c, colDef->constraints)
- {
- Constraint *con = lfirst_node(Constraint, c);
-
- if (con->contype == CONSTR_DEFAULT)
- {
- Node *expr;
- ParseState *pstate = make_parsestate(NULL);
-
- expr = transformExpr(pstate, copyObject(con->raw_expr), EXPR_KIND_COLUMN_DEFAULT);
- if (contain_volatile_functions(expr))
- {
- elog(ERROR,
- "Do not support replication of DDL statement that rewrites table using volatile functions");
- }
- }
- }
- }
- }
- break;
- }
case T_DropStmt:
{
DropStmt *dstmt = (DropStmt *) command->stmt;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index a66367fe7a..fd4140484c 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -413,6 +413,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -1368,9 +1369,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1404,6 +1418,14 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite &&
+ (!relentry->pubactions.pubddl_database || !relentry->pubactions.pubddl_table))
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1433,8 +1455,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1454,8 +1476,20 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
+
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1585,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
diff --git a/src/test/subscription/t/030_rep_ddls.pl b/src/test/subscription/t/030_rep_ddls.pl
index a03d598266..f5c50ffe88 100644
--- a/src/test/subscription/t/030_rep_ddls.pl
+++ b/src/test/subscription/t/030_rep_ddls.pl
@@ -488,13 +488,17 @@ is($result, qq(1), 'Only root table of the partitioned table is subscribed since
$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_subscription_rel where srrelid = 's1.test_part_a_1'::regclass::oid OR srrelid = 's1.test_part_a_2'::regclass::oid;");
is($result, qq(0), 'Leaf tables of the partitioned table are not subscribed since publish_via_partition_root is enabled');
-#TODO TEST certain DDLs are not replicated
-# Test DDL statement that rewrites table with volatile functions are not replicated
+# Test DDL statement that rewrites table with volatile functions are replicated with the same values from the publisher
$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD COLUMN volatile double precision DEFAULT 3 * random();");
-$result = $node_publisher->safe_psql('postgres', "SELECT count(*) FROM information_schema.columns WHERE table_name = 'test_rep' and column_name = 'volatile';");
-is($result, qq(1), 'Alter table add column default random() is executed on the publisher DB.');
+$result = $node_publisher->safe_psql('postgres', "SELECT avg(volatile) FROM test_rep;;");
+
+$node_publisher->wait_for_catchup('mysub');
-$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
+$result_sub = $node_subscriber->safe_psql('postgres', "SELECT avg(volatile) FROM test_rep;;");
+is($result, qq($result_sub), 'Alter table add column default random() is replicated correctly');
+
+#TODO TEST certain DDLs are not replicated
+#$result = $node_subscriber->wait_for_log("Do not support replication of DDL statement that rewrites table using volatile functions", $result);
pass "DDL replication tests passed!";
--
2.32.0
[application/octet-stream] 0011-Remove-non-transactional-ddl-message-decoding-becaus.patch (46.9K, ../../CAAD30U+wDPDFzUoPkSg2WYMNCXWNc8wa7GYB1Tzh_2PNUBsEHA@mail.gmail.com/13-0011-Remove-non-transactional-ddl-message-decoding-becaus.patch)
download | inline diff:
From 4e9706bf9421ec24d7fe2e4b985d44d456fa5e3e Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Tue, 24 May 2022 15:37:34 +0000
Subject: [PATCH 11/12] Remove non-transactional ddl message decoding because
the use case is unclear and it has unresolved issues under concurrency. Some
code cleanup.
---
.../test_decoding/expected/ddlmessages.out | 42 +--
contrib/test_decoding/test_decoding.c | 25 +-
.../access/rmgrdesc/logicalddlmsgdesc.c | 3 +-
src/backend/commands/tablecmds.c | 8 +-
src/backend/replication/logical/ddlmessage.c | 12 +-
src/backend/replication/logical/decode.c | 13 +-
src/backend/replication/logical/logical.c | 26 +-
src/backend/replication/logical/proto.c | 12 +-
.../replication/logical/reorderbuffer.c | 64 +---
src/backend/replication/logical/worker.c | 309 ++++++++++--------
src/backend/replication/pgoutput/pgoutput.c | 17 +-
src/backend/tcop/utility.c | 35 +-
src/include/replication/ddlmessage.h | 3 +-
src/include/replication/logicalproto.h | 4 +-
src/include/replication/output_plugin.h | 2 -
src/include/replication/reorderbuffer.h | 6 +-
16 files changed, 264 insertions(+), 317 deletions(-)
diff --git a/contrib/test_decoding/expected/ddlmessages.out b/contrib/test_decoding/expected/ddlmessages.out
index 823029d03d..56cbafa4cf 100644
--- a/contrib/test_decoding/expected/ddlmessages.out
+++ b/contrib/test_decoding/expected/ddlmessages.out
@@ -24,14 +24,14 @@ CREATE TABLE test_ddlmessage (id serial unique, data int);
ALTER TABLE test_ddlmessage add c3 varchar;
COMMIT;
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ data
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 70 content:CREATE TABLE test_ddlmessage (id serial unique primary key, data int);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE test_ddlmessage drop c3;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 27 content:DROP TABLE test_ddlmessage;
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 58 content:CREATE TABLE test_ddlmessage (id serial unique, data int);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 43 content:ALTER TABLE test_ddlmessage add c3 varchar;
(6 rows)
-- Test logging DDL in function
@@ -52,20 +52,20 @@ SELECT func_ddl ('tab_from_func');
(1 row)
SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'include-xids', '0', 'skip-empty-xacts', '1');
- data
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
- RETURNS VOID AS $$ +
- BEGIN +
- execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
- execute format('ALTER TABLE %I ADD c3 int', tname); +
- execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
- execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
- END; +
- $$ +
+ data
+--------------------------------------------------------------------------------------------------------------------------------------------------------------
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 375 content:CREATE OR REPLACE FUNCTION func_ddl (tname varchar(20)) +
+ RETURNS VOID AS $$ +
+ BEGIN +
+ execute format('CREATE TABLE %I(id int primary key, name varchar);', tname); +
+ execute format('ALTER TABLE %I ADD c3 int', tname); +
+ execute format('INSERT INTO %I VALUES (1, ''a'');', tname); +
+ execute format('INSERT INTO %I VALUES (2, ''b'', 22);', tname); +
+ END; +
+ $$ +
LANGUAGE plpgsql;
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
- DDL message: transactional: 1 prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 61 content:CREATE TABLE tab_from_func(id int primary key, name varchar);
+ DDL message: prefix: role: ddl_replication_user, search_path: "$user", public, sz: 36 content:ALTER TABLE tab_from_func ADD c3 int
BEGIN
table public.tab_from_func: INSERT: id[integer]:1 name[character varying]:'a' c3[integer]:null
table public.tab_from_func: INSERT: id[integer]:2 name[character varying]:'b' c3[integer]:22
diff --git a/contrib/test_decoding/test_decoding.c b/contrib/test_decoding/test_decoding.c
index a44e1f79e3..eb3dd76782 100644
--- a/contrib/test_decoding/test_decoding.c
+++ b/contrib/test_decoding/test_decoding.c
@@ -78,7 +78,7 @@ static void pg_decode_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pg_decode_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix,
+ const char *prefix,
const char *role, const char *search_path,
Size sz, const char *message);
static bool pg_decode_filter_prepare(LogicalDecodingContext *ctx,
@@ -123,7 +123,7 @@ static void pg_decode_stream_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix,
+ const char *prefix,
const char *role, const char *search_path,
Size sz, const char *message);
static void pg_decode_stream_truncate(LogicalDecodingContext *ctx,
@@ -771,13 +771,13 @@ pg_decode_message(LogicalDecodingContext *ctx,
static void
pg_decode_ddlmessage(LogicalDecodingContext *ctx,
- ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ ReorderBufferTXN *txn, XLogRecPtr lsn,
const char *prefix, const char *role, const char *search_path,
Size sz, const char *message)
{
OutputPluginPrepareWrite(ctx, true);
- appendStringInfo(ctx->out, "DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
- transactional, prefix, role, search_path, sz);
+ appendStringInfo(ctx->out, "DDL message: prefix: %s role: %s, search_path: %s, sz: %zu content:",
+ prefix, role, search_path, sz);
appendBinaryStringInfo(ctx->out, message, sz);
OutputPluginWrite(ctx, true);
}
@@ -989,23 +989,14 @@ pg_decode_stream_message(LogicalDecodingContext *ctx,
*/
static void
pg_decode_stream_ddlmessage(LogicalDecodingContext *ctx,
- ReorderBufferTXN *txn, XLogRecPtr lsn, bool transactional,
+ ReorderBufferTXN *txn, XLogRecPtr lsn,
const char *prefix, const char * role, const char * search_path,
Size sz, const char *message)
{
OutputPluginPrepareWrite(ctx, true);
- if (transactional)
- {
- appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu",
- transactional, prefix, role, search_path, sz);
- }
- else
- {
- appendStringInfo(ctx->out, "streaming DDL message: transactional: %d prefix: %s role: %s, search_path: %s, sz: %zu content:",
- transactional, prefix, role, search_path, sz);
- appendBinaryStringInfo(ctx->out, message, sz);
- }
+ appendStringInfo(ctx->out, "streaming DDL message: prefix: %s role: %s, search_path: %s, sz: %zu",
+ prefix, role, search_path, sz);
OutputPluginWrite(ctx, true);
}
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
index 7a352d540a..0aaebf8a08 100644
--- a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -32,8 +32,7 @@ logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
Assert(prefix[xlrec->prefix_size] != '\0');
- appendStringInfo(buf, "%s, prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
- xlrec->transactional ? "transactional" : "non-transactional",
+ appendStringInfo(buf, "prefix \"%s\"; role \"%s\"; search_path \"%s\"; payload (%zu bytes): ",
prefix, role, search_path, xlrec->message_size);
/* Write message payload as a series of hex bytes */
for (int cnt = 0; cnt < xlrec->message_size; cnt++)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 65386f2641..94e350b80d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -1516,13 +1516,11 @@ RemoveRelations(ParseState *pstate, DropStmt *drop, bool isCompleteQuery)
/* Log the Drop command for logical replication */
if (ddlxlog)
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
pstate->p_sourcetext,
- strlen(pstate->p_sourcetext),
- transactional);
+ strlen(pstate->p_sourcetext));
}
performMultipleDeletions(objects, drop->behavior, flags);
@@ -3876,13 +3874,11 @@ RenameRelation(ParseState *pstate, RenameStmt *stmt, bool isCompleteQuery)
if (ddlxlog &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
pstate->p_sourcetext,
- strlen(pstate->p_sourcetext),
- transactional);
+ strlen(pstate->p_sourcetext));
}
/* Do the work */
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
index f93573079a..5f89afec49 100644
--- a/src/backend/replication/logical/ddlmessage.c
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -47,7 +47,7 @@
*/
XLogRecPtr
LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
- size_t size, bool transactional)
+ size_t size)
{
xl_logical_ddl_message xlrec;
const char *role;
@@ -55,16 +55,12 @@ LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *message,
role = GetUserNameFromId(roleoid, false);
/*
- * Force xid to be allocated if we're emitting a transactional message.
+ * Force xid to be allocated since we're emitting a transactional message.
*/
- if (transactional)
- {
- Assert(IsTransactionState());
- GetCurrentTransactionId();
- }
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
xlrec.dbId = MyDatabaseId;
- xlrec.transactional = transactional;
/* trailing zero is critical; see logicalddlmsg_desc */
xlrec.prefix_size = strlen(prefix) + 1;
xlrec.role_size = strlen(role) + 1;
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index 034c7f2413..ce5b595326 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -615,7 +615,6 @@ logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
TransactionId xid = XLogRecGetXid(r);
uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
RepOriginId origin_id = XLogRecGetOrigin(r);
- Snapshot snapshot;
xl_logical_ddl_message *message;
if (info != XLOG_LOGICAL_DDL_MESSAGE)
@@ -637,17 +636,7 @@ logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
FilterByOrigin(ctx, origin_id))
return;
- if (message->transactional &&
- !SnapBuildProcessChange(builder, xid, buf->origptr))
- return;
- else if (!message->transactional &&
- (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT ||
- SnapBuildXactNeedsSkip(builder, buf->origptr)))
- return;
-
- snapshot = SnapBuildGetOrBuildSnapshot(builder, xid);
- ReorderBufferQueueDDLMessage(ctx->reorder, xid, snapshot, buf->endptr,
- message->transactional,
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
message->message,
/* first part of message is prefix */
message->message + message->prefix_size,
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 3004f02433..c02ea6fb99 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -74,8 +74,8 @@ static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role, const char *search_path,
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
Size message_size, const char *message);
/* streaming callbacks */
@@ -95,8 +95,8 @@ static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *tx
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role, const char *search_path,
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -1233,10 +1233,9 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role,
- const char *search_path, Size message_size,
- const char *message)
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char *search_path,
+ Size message_size, const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
LogicalErrorCallbackState state;
@@ -1262,7 +1261,7 @@ ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix,
role, search_path, message_size, message);
/* Pop the error context stack */
@@ -1586,10 +1585,9 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void
stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
- const char *prefix, const char *role,
- const char* search_path, Size message_size,
- const char *message)
+ XLogRecPtr message_lsn, const char *prefix,
+ const char *role, const char* search_path,
+ Size message_size, const char *message)
{
LogicalDecodingContext *ctx = cache->private_data;
LogicalErrorCallbackState state;
@@ -1619,7 +1617,7 @@ stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
ctx->write_location = message_lsn;
/* do the actual work: call callback */
- ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, transactional, prefix,
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix,
role, search_path, message_size, message);
/* Pop the error context stack */
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index 3cfa94dd8c..2072207647 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -670,16 +670,12 @@ logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
const char **prefix,
const char **role,
const char **search_path,
- bool *transactional,
Size *sz)
{
- uint8 flags;
const char *msg;
//TODO double check when do we need to get TransactionId.
- flags = pq_getmsgint(in, 1);
- *transactional = (flags & MESSAGE_TRANSACTIONAL) > 0;
*lsn = pq_getmsgint64(in);
*prefix = pq_getmsgstring(in);
*role = pq_getmsgstring(in);
@@ -695,22 +691,16 @@ logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
*/
void
logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message)
{
- uint8 flags = 0;
pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
- /* encode and send message flags */
- if (transactional)
- flags |= MESSAGE_TRANSACTIONAL;
-
/* transaction ID (if not valid, we're not streaming) */
if (TransactionIdIsValid(xid))
pq_sendint32(out, xid);
- pq_sendint8(out, flags);
pq_sendint64(out, lsn);
pq_sendstring(out, prefix);
pq_sendstring(out, role);
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index ca01336604..f5a3247348 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -881,61 +881,33 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
- * A transactional DDL message is queued to be processed upon commit and a
- * non-transactional DDL message gets processed immediately.
+ * A transactional DDL message is queued to be processed upon commit
*/
void
ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
- Snapshot snapshot, XLogRecPtr lsn,
- bool transactional, const char *prefix,
+ XLogRecPtr lsn, const char *prefix,
const char *role, const char *search_path,
Size message_size, const char *message)
{
- if (transactional)
- {
- MemoryContext oldcontext;
- ReorderBufferChange *change;
-
- Assert(xid != InvalidTransactionId);
-
- oldcontext = MemoryContextSwitchTo(rb->context);
-
- change = ReorderBufferGetChange(rb);
- change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
- change->data.ddlmsg.prefix = pstrdup(prefix);
- change->data.ddlmsg.role = pstrdup(role);
- change->data.ddlmsg.search_path = pstrdup(search_path);
- change->data.ddlmsg.message_size = message_size;
- change->data.ddlmsg.message = palloc(message_size);
- memcpy(change->data.ddlmsg.message, message, message_size);
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
- ReorderBufferQueueChange(rb, xid, lsn, change, false);
+ Assert(xid != InvalidTransactionId);
- MemoryContextSwitchTo(oldcontext);
- }
- else
- {
- ReorderBufferTXN *txn = NULL;
- volatile Snapshot snapshot_now = snapshot;
+ oldcontext = MemoryContextSwitchTo(rb->context);
- if (xid != InvalidTransactionId)
- txn = ReorderBufferTXNByXid(rb, xid, true, NULL, lsn, true);
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.role = pstrdup(role);
+ change->data.ddlmsg.search_path = pstrdup(search_path);
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
- /* setup snapshot to allow catalog access */
- SetupHistoricSnapshot(snapshot_now, NULL);
- PG_TRY();
- {
- rb->ddlmessage(rb, txn, lsn, false, prefix, role, search_path, message_size, message);
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
- TeardownHistoricSnapshot(false);
- }
- PG_CATCH();
- {
- TeardownHistoricSnapshot(true);
- PG_RE_THROW();
- }
- PG_END_TRY();
- }
+ MemoryContextSwitchTo(oldcontext);
}
/*
@@ -2037,14 +2009,14 @@ ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferChange *change, bool streaming)
{
if (streaming)
- rb->stream_ddlmessage(rb, txn, change->lsn, true,
+ rb->stream_ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
change->data.ddlmsg.role,
change->data.ddlmsg.search_path,
change->data.ddlmsg.message_size,
change->data.ddlmsg.message);
else
- rb->ddlmessage(rb, txn, change->lsn, true,
+ rb->ddlmessage(rb, txn, change->lsn,
change->data.ddlmsg.prefix,
change->data.ddlmsg.role,
change->data.ddlmsg.search_path,
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 1108030179..ef79d10115 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2464,14 +2464,13 @@ static void
apply_handle_ddlmessage(StringInfo s)
{
XLogRecPtr lsn;
- bool transactional;
Size sz;
const char *prefix;
const char *role;
const char *search_path;
const char *msg;
- msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &transactional, &sz);
+ msg = logicalrep_read_ddlmessage(s, &lsn, &prefix, &role, &search_path, &sz);
apply_execute_sql_command(msg, role, search_path, true);
}
@@ -2486,98 +2485,41 @@ execute_sql_command_error_cb(void *arg)
}
/*
- * Execute an SQL command. This can be multiple queries.
- * This is modified based on pglogical_execute_sql_command().
+ * Preprocess certain DDL commands before apply
+ * -Remove data population for table creation
+ * -Enable missing_ok for drop stmt
+ * -Disallow table rewrites using volatile functions
*/
static void
-apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
- bool isTopLevel)
+preprocess_ddl(RawStmt *command, char **schemaname, char **relname, bool *is_partitioned_table)
{
- const char *save_debug_query_string = debug_query_string;
- List *parsetree_list;
- ListCell *parsetree_item;
- MemoryContext oldcontext;
- ErrorContextCallback errcallback;
- int save_nestlevel;
-
- /*
- * Switch to appropriate context for constructing parsetrees.
- */
- oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
- begin_replication_step();
-
- /*
- * Set the current role to the user that executed the command on the
- * publication server.
- * Set the current search_path to the search_path on the publication
- * server when the command was executed.
- */
- save_nestlevel = NewGUCNestLevel();
- SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
- SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
-
- errcallback.callback = execute_sql_command_error_cb;
- errcallback.arg = (char *) cmdstr;
- errcallback.previous = error_context_stack;
- error_context_stack = &errcallback;
-
- debug_query_string = cmdstr;
-
- parsetree_list = pg_parse_query(cmdstr);
-
- /*
- * Do a limited amount of safety checking against CONCURRENTLY commands
- * executed in situations where they aren't allowed. The sender side should
- * provide protection, but better be safe than sorry.
- */
- isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
-
- /*
- * Switch back to transaction context to enter the loop.
- */
- MemoryContextSwitchTo(oldcontext);
-
- foreach(parsetree_item, parsetree_list)
+ switch(nodeTag(command->stmt))
{
- List *plantree_list;
- List *querytree_list;
- RawStmt *command = (RawStmt *) lfirst(parsetree_item);
- CommandTag commandTag;
- MemoryContext per_parsetree_context = NULL;
- Portal portal;
- DestReceiver *receiver;
- bool snapshot_set = false;
- char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
- char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
- bool is_partitioned_table = false;
-
- commandTag = CreateCommandTag((Node *)command);
-
- /* The following DDL commands need special handling */
-
- /*
- * Remember the schemaname and relname if the cmd is going to create a table
- * because we will need them for some post-processing after we
- * execute the stmt. At that point, command->stmt may have been freeed up.
- */
- if (commandTag == CMDTAG_CREATE_TABLE)
+ case T_CreateStmt:
{
+ /*
+ * Remember the schemaname and relname if the cmd is going to create a table
+ * because we will need them for some post-processing after we
+ * execute the stmt. At that point, command->stmt may have been freeed up.
+ */
CreateStmt *cstmt = (CreateStmt *) command->stmt;
RangeVar *rv = cstmt->relation;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
if (cstmt->inhRelations != NIL || cstmt->partspec != NULL)
- is_partitioned_table = true;
+ *is_partitioned_table = true;
+
+ break;
}
- else if (commandTag == CMDTAG_CREATE_TABLE_AS)
+ case T_CreateTableAsStmt:
{
CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
if (castmt->objtype == OBJECT_TABLE)
{
RangeVar *rv = castmt->into->rel;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
/*
* Force skipping data population to avoid data inconsistency.
@@ -2585,17 +2527,18 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
*/
castmt->into->skipData = true;
}
+ break;
}
/* SELECT INTO */
- else if (commandTag == CMDTAG_SELECT)
+ case T_SelectStmt:
{
SelectStmt *sstmt = (SelectStmt *) command->stmt;
if (sstmt->intoClause != NULL)
{
RangeVar *rv = sstmt->intoClause->rel;
- schemaname = rv->schemaname;
- relname = rv->relname;
+ *schemaname = rv->schemaname;
+ *relname = rv->relname;
/*
* Force skipping data population to avoid data inconsistency.
@@ -2603,13 +2546,14 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
*/
sstmt->intoClause->skipData = true;
}
+ break;
}
/*
* ALTER TABLE ADD COLUMN col DEFAULT volatile_expr is not supported.
* Until we support logical replication of table rewrite, see ATRewriteTables()
* for details on table rewrite.
*/
- else if (commandTag == CMDTAG_ALTER_TABLE)
+ case T_AlterTableStmt:
{
AlterTableStmt *atstmt = (AlterTableStmt *) command->stmt;
ListCell *lc;
@@ -2637,15 +2581,153 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
if (contain_volatile_functions(expr))
{
elog(ERROR,
- "Do not support replication of DDL statement that rewrites table using volatile functions: %s",
- cmdstr);
+ "Do not support replication of DDL statement that rewrites table using volatile functions");
}
}
}
}
}
+ break;
+ }
+ case T_DropStmt:
+ {
+ DropStmt *dstmt = (DropStmt *) command->stmt;
+ dstmt->missing_ok = true;
+ break;
+ }
+ default:
+ break;
+ }
+}
+
+/*
+* Table created by DDL replication (database level) is automatically
+* added to the subscription here.
+*
+* Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
+* command to set the relstate to SUBREL_STATE_INIT so DML changes on this
+* new table can be replicated without having to manually run
+* "alter subscription ... refresh publication"
+*/
+static void
+handle_create_table(char* relname, char* schemaname, bool is_partitioned_table)
+{
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ {
+ /*
+ * Try to resolve unqualified relname.
+ * Notice we have set the search_path to the original search_path on the publisher
+ * at the beginning of this function.
+ */
+ relid = RelnameGetRelid(relname);
+ }
+
+ if (relid != InvalidOid)
+ {
+ bool subscribe_table = true;
+
+ if (is_partitioned_table)
+ {
+ Relation rel = RelationIdGetRelation(relid);
+ char *table_name = RelationGetRelationName(rel);
+ char *schema_name = get_namespace_name(RelationGetNamespace(rel));
+ /*
+ * Connect to the source DB and check whehter the partitioned table should be subscribed.
+ * Because it depends on the setting of publish_via_partition_root, which the subscription
+ * doesn't know.
+ */
+ subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
+ RelationClose(rel);
}
+ if (subscribe_table)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_INIT,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+ }
+}
+
+/*
+ * Execute an SQL command. This can be multiple queries.
+ * This is modified based on pglogical_execute_sql_command().
+ */
+static void
+apply_execute_sql_command(const char *cmdstr, const char *role, const char *search_path,
+ bool isTopLevel)
+{
+ const char *save_debug_query_string = debug_query_string;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ MemoryContext oldcontext;
+ ErrorContextCallback errcallback;
+ int save_nestlevel;
+
+ /*
+ * Switch to appropriate context for constructing parsetrees.
+ */
+ oldcontext = MemoryContextSwitchTo(ApplyMessageContext);
+ begin_replication_step();
+
+ /*
+ * Set the current role to the user that executed the command on the
+ * publication server.
+ * Set the current search_path to the search_path on the publication
+ * server when the command was executed.
+ */
+ save_nestlevel = NewGUCNestLevel();
+ SetConfigOption("role", role, PGC_INTERNAL, PGC_S_OVERRIDE);
+ SetConfigOption("search_path", search_path, PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ errcallback.callback = execute_sql_command_error_cb;
+ errcallback.arg = (char *) cmdstr;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ debug_query_string = cmdstr;
+
+ parsetree_list = pg_parse_query(cmdstr);
+
+ /*
+ * Do a limited amount of safety checking against CONCURRENTLY commands
+ * executed in situations where they aren't allowed. The sender side should
+ * provide protection, but better be safe than sorry.
+ */
+ isTopLevel = isTopLevel && (list_length(parsetree_list) == 1);
+
+ /*
+ * Switch back to transaction context to enter the loop.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ DestReceiver *receiver;
+ bool snapshot_set = false;
+ char *schemaname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ char *relname = NULL; /* For CREATE TABLE and CREATE TABLE AS stmt only */
+ bool is_partitioned_table = false;
+
+ commandTag = CreateCommandTag((Node *)command);
+ preprocess_ddl(command, &schemaname, &relname, &is_partitioned_table);
+
/*
* Set up a snapshot if parse analysis/planning will need one.
*/
@@ -2739,63 +2821,8 @@ apply_execute_sql_command(const char *cmdstr, const char *role, const char *sear
CommandCounterIncrement();
- /*
- * Table created by DDL replication (database level) is automatically
- * added to the subscription here.
- *
- * Call AddSubscriptionRelState for CREATE TABEL and CREATE TABLE AS
- * command to set the relstate to SUBREL_STATE_INIT so DML changes on this
- * new table can be replicated without having to manually run
- * "alter subscription ... refresh publication"
- */
if (relname != NULL)
- {
- Oid relid;
- Oid relnamespace = InvalidOid;
-
- if (schemaname != NULL)
- relnamespace = get_namespace_oid(schemaname, false);
- if (relnamespace != InvalidOid)
- relid = get_relname_relid(relname, relnamespace);
- else
- {
- /*
- * Try to resolve unqualified relname.
- * Notice we have set the search_path to the original search_path on the publisher
- * at the beginning of this function.
- */
- relid = RelnameGetRelid(relname);
- }
-
- if (relid != InvalidOid)
- {
- bool subscribe_table = true;
-
- if (is_partitioned_table)
- {
- Relation rel = RelationIdGetRelation(relid);
- char *table_name = RelationGetRelationName(rel);
- char *schema_name = get_namespace_name(RelationGetNamespace(rel));
- /*
- * Connect to the source DB and check whehter the partitioned table should be subscribed.
- * Because it depends on the setting of publish_via_partition_root, which the subscription
- * doesn't know.
- */
- subscribe_table = IsPartitionedTablePublishedOnSource(MySubscription, schema_name, table_name);
- RelationClose(rel);
- }
-
- if (subscribe_table)
- {
- AddSubscriptionRelState(MySubscription->oid, relid,
- SUBREL_STATE_INIT,
- InvalidXLogRecPtr);
- ereport(DEBUG1,
- (errmsg_internal("table \"%s\" added to subscription \"%s\"",
- relname, MySubscription->name)));
- }
- }
- }
+ handle_create_table(relname, schemaname, is_partitioned_table);
}
/*
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index db221c7e8d..a66367fe7a 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -56,7 +56,7 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
Size sz, const char *message);
static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
@@ -1705,13 +1705,14 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional,
+ XLogRecPtr message_lsn,
const char *prefix, const char * role,
const char *search_path, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
ListCell *lc;
+ PGOutputTxnData *txndata;
/* Reload publications if needed before use. */
if (!publications_valid)
@@ -1745,20 +1746,16 @@ pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
* Output BEGIN if we haven't yet. Avoid for non-transactional
* messages.
*/
- if (transactional)
- {
- PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ txndata = (PGOutputTxnData *) txn->output_plugin_private;
- /* Send BEGIN if we haven't yet */
- if (txndata && !txndata->sent_begin_txn)
- pgoutput_send_begin(ctx, txn);
- }
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
OutputPluginPrepareWrite(ctx, true);
logicalrep_write_ddlmessage(ctx->out,
xid,
message_lsn,
- transactional,
prefix,
role,
search_path,
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 1abd77a60e..7cf8fe0ba3 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1131,13 +1131,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
break;
@@ -1173,13 +1171,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
default:
break;
@@ -1195,18 +1191,25 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
case T_AlterTableStmt:
case T_IndexStmt:
+ break;
+ /*
+ * Rename of objects other than table is only allowed in database level
+ * replication.
+ * Rename of table is allowed in both table level and database level
+ * replication.
+ */
case T_RenameStmt:
{
RenameStmt *stmt = (RenameStmt *) parsetree;
- if(!stmt->relation && ddl_need_xlog(InvalidOid, true)){
- bool transactional = true;
+ if(!stmt->relation && ddl_need_xlog(InvalidOid, true))
+ {
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
+ break;
}
case T_AlterOwnerStmt: /* TODO, it is data control case, save for later update */
break;
@@ -1236,13 +1239,11 @@ LogLogicalDDLCommand(Node *parsetree, const char *queryString)
*/
if (ddl_need_xlog(InvalidOid, true))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
break;
}
@@ -1546,13 +1547,11 @@ ProcessUtilitySlow(ParseState *pstate,
isCompleteQuery &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
/* ... and do it */
@@ -1783,13 +1782,11 @@ ProcessUtilitySlow(ParseState *pstate,
isCompleteQuery &&
ddl_need_xlog(relid, false))
{
- bool transactional = true;
const char* prefix = "";
LogLogicalDDLMessage(prefix,
GetUserId(),
queryString,
- strlen(queryString),
- transactional);
+ strlen(queryString));
}
address =
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
index 1e8ef22296..0418d16dca 100644
--- a/src/include/replication/ddlmessage.h
+++ b/src/include/replication/ddlmessage.h
@@ -20,7 +20,6 @@
typedef struct xl_logical_ddl_message
{
Oid dbId; /* database Oid emitted from */
- bool transactional; /* is message transactional? */
Size prefix_size; /* length of prefix */
Size role_size; /* length of the role that executes the DDL command */
Size search_path_size; /* length of the search path */
@@ -36,7 +35,7 @@ typedef struct xl_logical_ddl_message
#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid roleoid, const char *ddl_message,
- size_t size, bool transactional);
+ size_t size);
/* RMGR API*/
#define XLOG_LOGICAL_DDL_MESSAGE 0x00
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index 28ff562d62..cc0ba60905 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -233,11 +233,11 @@ extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecP
bool transactional, const char *prefix,
Size sz, const char *message);
extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+ const char *prefix, const char *role,
const char *search_path, Size sz, const char *message);
extern const char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix,
const char **role, const char **search_path,
- bool *transactional, Size *sz);
+ Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 5b1c245b72..444b75bef0 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -94,7 +94,6 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -219,7 +218,6 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index dd89e08efc..cd8c69ceb8 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -445,7 +445,6 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -523,7 +522,6 @@ typedef void (*ReorderBufferStreamDDLMessageCB) (
ReorderBuffer *rb,
ReorderBufferTXN *txn,
XLogRecPtr message_lsn,
- bool transactional,
const char *prefix,
const char *role,
const char *search_path,
@@ -671,8 +669,8 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
-extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
- bool transactional, const char *prefix, const char *role,
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, const char *role,
const char *search_path, Size message_size, const char *message);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-20 03:31 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-20 03:31 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Saturday, June 18, 2022 3:38 AM Zheng Li <zhengli10@gmail.com> wrote:
> On Wed, Jun 15, 2022 at 12:00 AM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> >
> > On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> > >
> > >
> > > While I agree that the deparser is needed to handle the potential
> > > syntax differences between the pub/sub, I think it's only relevant
> > > for the use cases where only a subset of tables in the database are
> > > replicated. For other use cases where all tables, functions and
> > > other objects need to be replicated, (for example, creating a
> > > logical replica for major version upgrade) there won't be any syntax
> > > difference to handle and the schemas are supposed to match exactly
> > > between the pub/sub. In other words the user seeks to create an
> > > identical replica of the source database and the DDLs should be
> > > replicated as is in this case.
> > >
> >
> > I think even for database-level replication we can't assume that
> > source and target will always have the same data in which case "Create
> > Table As ..", "Alter Table .. " kind of statements can't be replicated
> > as it is because that can lead to different results.
> "Create Table As .." is already handled by setting the skipData flag of the
> statement parsetreee before replay:
>
> /*
> * Force skipping data population to avoid data inconsistency.
> * Data should be replicated from the publisher instead.
> */
> castmt->into->skipData = true;
>
> "Alter Table .. " that rewrites with volatile expressions can also be handled
> without any syntax change, by enabling the table rewrite replication and
> converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> I've also adopted this approach in my latest patch
> 0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch
>
> > The other point
> > is tomorrow we can extend the database level option/syntax to exclude
> > a few objects (something like [1]) as well in which case we again need
> > to filter at the publisher level
>
> I think for such cases it's not full database replication and we could treat it as
> table level DDL replication, i.e. use the the deparser format.
Hi,
Here are some points in my mind about the two approaches discussed here.
1) search_patch vs schema qualify
Again, I still think it will bring more flexibility and security by schema qualify the
objects in DDL command as mentioned before[1].
Besides, a schema qualified DDL is also more appropriate for other use
cases(e.g. a table-level replication). As it's possible the schema is different
between pub/sub and it's easy to cause unexpected and undetectable failure if
we just log the search_path.
It makes more sense to me to have the same style WAL log(schema qualified) for
both database level or table level replication as it will bring more
flexibility.
> "Create Table As .." is already handled by setting the skipData flag of the
> statement parsetreee before replay:
2) About the handling of CREATE TABLE AS:
I think it's not a appropriate approach to set the skipdata flag on subscriber
as it cannot handle EXECUTE command in CTAS.
CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
The Prepared statement is a temporary object which we don't replicate. So if
you directly execute the original SQL on subscriber, even if you set skipdata
it will fail.
I think it difficult to make this work as you need handle the create/drop of
this prepared statement. And even if we extended subscriber's code to make it
work, it doesn't seems like a standard and elegant approach.
> "Alter Table .. " that rewrites with volatile expressions can also be handled
> without any syntax change, by enabling the table rewrite replication and
> converting the rewrite inserts to updates. ZJ's patch introduced this solution.
3) About the handling of ALTER TABLE rewrite.
The approach I proposed before is based on the event trigger + deparser
approach. We were able to improve that approach as we don't need to replicate
the rewrite in many cases. For example: we don't need to replicate rewrite dml
if there is no volatile/mutable function. We should check and filter these case
at publisher (e.g. via deparser) instead of checking that at subscriber.
Besides, as discussed, we need to give warning or error for the cases when DDL
contains volatile function which would be executed[2]. We should check this at
publisher as well(via deparser).
> I think for such cases it's not full database replication and we could treat it as
> table level DDL replication, i.e. use the the deparser format.
4) I think the point could be that we should make the WAL log format extendable
so that we can extend it to support more useful feature(table filter/schema
maps/DDL filter). If we just WAL log the original SQL, it seems it's difficult
to extend it in the future ?
[1] https://www.postgresql.org/message-id/202204081134.6tcmf5cxl3sz%40alvherre.pgsql
[2] https://www.postgresql.org/message-id/CAA4eK1JVynFsj%2BmcRWj9sewR2yNUs6LuNxJ0eN-gNJ83oKcUOQ%40mail.g...
Best regards,
Hou zj
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-21 12:19 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 3 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-21 12:19 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Monday, June 20, 2022 11:32 AM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
>
> On Saturday, June 18, 2022 3:38 AM Zheng Li <zhengli10@gmail.com> wrote:
> > On Wed, Jun 15, 2022 at 12:00 AM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > >
> > > On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> > > >
> > > >
> > > > While I agree that the deparser is needed to handle the potential
> > > > syntax differences between the pub/sub, I think it's only relevant
> > > > for the use cases where only a subset of tables in the database are
> > > > replicated. For other use cases where all tables, functions and
> > > > other objects need to be replicated, (for example, creating a
> > > > logical replica for major version upgrade) there won't be any syntax
> > > > difference to handle and the schemas are supposed to match exactly
> > > > between the pub/sub. In other words the user seeks to create an
> > > > identical replica of the source database and the DDLs should be
> > > > replicated as is in this case.
> > > >
> > >
> > > I think even for database-level replication we can't assume that
> > > source and target will always have the same data in which case "Create
> > > Table As ..", "Alter Table .. " kind of statements can't be replicated
> > > as it is because that can lead to different results.
> > "Create Table As .." is already handled by setting the skipData flag of the
> > statement parsetreee before replay:
> >
> > /*
> > * Force skipping data population to avoid data inconsistency.
> > * Data should be replicated from the publisher instead.
> > */
> > castmt->into->skipData = true;
> >
> > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > without any syntax change, by enabling the table rewrite replication and
> > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> > I've also adopted this approach in my latest patch
> > 0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch
> >
> > > The other point
> > > is tomorrow we can extend the database level option/syntax to exclude
> > > a few objects (something like [1]) as well in which case we again need
> > > to filter at the publisher level
> >
> > I think for such cases it's not full database replication and we could treat it as
> > table level DDL replication, i.e. use the the deparser format.
>
> Hi,
>
> Here are some points in my mind about the two approaches discussed here.
>
> 1) search_patch vs schema qualify
>
> Again, I still think it will bring more flexibility and security by schema qualify the
> objects in DDL command as mentioned before[1].
>
> Besides, a schema qualified DDL is also more appropriate for other use
> cases(e.g. a table-level replication). As it's possible the schema is different
> between pub/sub and it's easy to cause unexpected and undetectable failure if
> we just log the search_path.
>
> It makes more sense to me to have the same style WAL log(schema qualified)
> for
> both database level or table level replication as it will bring more
> flexibility.
>
>
> > "Create Table As .." is already handled by setting the skipData flag of the
> > statement parsetreee before replay:
>
> 2) About the handling of CREATE TABLE AS:
>
> I think it's not a appropriate approach to set the skipdata flag on subscriber
> as it cannot handle EXECUTE command in CTAS.
>
> CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
>
> The Prepared statement is a temporary object which we don't replicate. So if
> you directly execute the original SQL on subscriber, even if you set skipdata
> it will fail.
>
> I think it difficult to make this work as you need handle the create/drop of
> this prepared statement. And even if we extended subscriber's code to make it
> work, it doesn't seems like a standard and elegant approach.
>
>
> > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > without any syntax change, by enabling the table rewrite replication and
> > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
>
> 3) About the handling of ALTER TABLE rewrite.
>
> The approach I proposed before is based on the event trigger + deparser
> approach. We were able to improve that approach as we don't need to
> replicate
> the rewrite in many cases. For example: we don't need to replicate rewrite dml
> if there is no volatile/mutable function. We should check and filter these case
> at publisher (e.g. via deparser) instead of checking that at subscriber.
>
> Besides, as discussed, we need to give warning or error for the cases when DDL
> contains volatile function which would be executed[2]. We should check this at
> publisher as well(via deparser).
>
>
> > I think for such cases it's not full database replication and we could treat it as
> > table level DDL replication, i.e. use the the deparser format.
>
> 4) I think the point could be that we should make the WAL log format
> extendable
> so that we can extend it to support more useful feature(table filter/schema
> maps/DDL filter). If we just WAL log the original SQL, it seems it's difficult
> to extend it in the future ?
Attach the new version patch set which added support for CREATE/DROP/ATER
Sequence and CREATE/DROP Schema ddl commands which are provided by Ajin
Cherian off list.
The new version patch will also check function's volatility[1] in ALTER TABLE
command. If any function to be executed is volatile, we report an ERROR.
Whether WARNING is better to be used here is still under consideration.
[1] https://www.postgresql.org/message-id/CAA4eK1JVynFsj%2BmcRWj9sewR2yNUs6LuNxJ0eN-gNJ83oKcUOQ%40mail.g...
Best regards,
Hou zj
Attachments:
[application/octet-stream] v9-0002-Support-DDL-replication.patch (123.7K, ../../OS0PR01MB5716B1526C2EDA66907E733B94B39@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v9-0002-Support-DDL-replication.patch)
download | inline diff:
From ce9c54542b3c90bbd125f7c20ce89128c0089a84 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 21 Jun 2022 17:41:27 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE] command and
- CREATE [TABLE|SEQUENCE|SCHEMA] command:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER SCHEMA command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 168 +++++++++-
src/backend/commands/publicationcmds.c | 97 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
33 files changed, 1536 insertions(+), 217 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 8c7fca6..20bf8c1 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..c3d9aaa 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,164 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ command = deparse_drop_table(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_TableDropEnd,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "sequence", 8) == 0)
+ {
+ command = deparse_drop_sequence(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "schema", 6) == 0)
+ {
+ command = deparse_drop_schema(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..c36b8a7 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,46 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1116,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..74108b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5218,7 +5218,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 38e3b1c..2c667bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57..331949c 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2156,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2215,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2329,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2515,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0e8fda9..27de475 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index d1ae699..7171768 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..bb98c17 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..de58035 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v9-0001-Functions-to-deparse-DDL-commands.patch (103.0K, ../../OS0PR01MB5716B1526C2EDA66907E733B94B39@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v9-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From f0d8dc26d568c5b73f7ed9d1d8d9d4eb0155d38b Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/DROP SCHEMA
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 2636 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++++
src/backend/commands/sequence.c | 34 +
src/backend/utils/adt/ruleutils.c | 9 +
src/include/catalog/pg_proc.dat | 7 +-
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 12 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/ruleutils.h | 1 +
10 files changed, 3451 insertions(+), 1 deletion(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..7b180bd
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,2636 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "lib/ilist.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s %{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (saw_notnull)
+ column = new_objtree_VA("%{name}I WITH OPTIONS NOT NULL", 2,
+ "type", ObjTypeString, "column_notnull",
+ "name", ObjTypeString, coldef->colname);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ {
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ append_array_object(tmp, "parents", parents);
+ }
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+ tmp = new_objtree_VA("CACHE %{value}s",
+ 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("%{no}s CYCLE",
+ 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ tmp = new_objtree_VA("INCREMENT BY %{value}s",
+ 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ tmp = new_objtree_VA("MINVALUE %{value}s",
+ 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ tmp = new_objtree_VA("MAXVALUE %{value}s",
+ 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ tmp = new_objtree_VA("START WITH %{value}s",
+ 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ tmp = new_objtree_VA("RESTART %{value}s",
+ 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+/*
+ * deparse_CreateSchemaStmt
+ * deparse a CreateSchemaStmt
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ createSchema =
+ new_objtree_VA("CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s",
+ 2,
+ "name", ObjTypeString, node->schemaname,
+ "if_not_exists", ObjTypeString,
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+ auth = new_objtree_VA("AUTHORIZATION %{authorization_role}I", 0);
+ if (node->authrole)
+ append_string_object(auth, "authorization_role",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "authorization_role");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "authorization", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/* deparse_CreateSeqStmt
+ * deparse a CreateSeqStmt
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ Form_pg_sequence_data seqdata = get_sequence_values(objectId);
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ createSeq =
+ new_objtree_VA("CREATE %{persistence}s SEQUENCE %{identity}D "
+ "%{definition: }s",
+ 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "identity", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * deparse_AlterSeqStmt
+ * deparse an AlterSeqStmt
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+
+ seqdata = get_sequence_values(objectId);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq =
+ new_objtree_VA("ALTER SEQUENCE %{identity}D %{definition: }s", 0);
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "identity", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_sequence(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SEQUENCE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_schema(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SCHEMA IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index ddf219b..ce16206 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1699,6 +1699,40 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ /* open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..2c4bcdc 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -2143,6 +2143,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 9da2300..2875106 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..4f3c55f
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,12 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_sequence(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_schema(const char *objidentity, DropBehavior behavior);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..467de7c 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -29,6 +29,7 @@ extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
--
2.7.2.windows.1
[application/octet-stream] v9-0003-support-CREATE-TABLE-ASSELECT-INTO.patch (14.1K, ../../OS0PR01MB5716B1526C2EDA66907E733B94B39@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v9-0003-support-CREATE-TABLE-ASSELECT-INTO.patch)
download | inline diff:
From ca110ee2206fd9ea438b9d6e4cd14213ea888a11 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 21 Jun 2022 16:28:22 +0800
Subject: [PATCH] support CREATE TABLE ASSELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 +++
src/backend/commands/ddl_deparse.c | 18 ++++
src/backend/commands/event_trigger.c | 153 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 207 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index d853d41..94d3aaa 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -2439,6 +2439,21 @@ removed feature
return alterTableStmt;
}
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
/*
* Handle deparsing of simple commands.
*
@@ -2595,6 +2610,9 @@ deparse_utility_command(CollectedCommand *cmd)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c3d9aaa..b3f1b9b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -586,7 +588,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -869,6 +872,144 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1149,7 +1290,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1873,6 +2015,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1890,6 +2033,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index c36b8a7..eda5995 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -963,6 +963,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_DROP_SCHEMA
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -974,6 +979,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..4d003de 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index 3a9c9f0..467cd65 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bb98c17..5fe28c9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11899,4 +11899,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v9-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB5716B1526C2EDA66907E733B94B39@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v9-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From b25920a8213e38de9eb8bb096bdc29d64f3e9482 Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-22 05:38 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-06-22 05:38 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Jun 15, 2022 at 1:00 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> >
> >
> > While I agree that the deparser is needed to handle the potential
> > syntax differences between
> > the pub/sub, I think it's only relevant for the use cases where only a
> > subset of tables in the database
> > are replicated. For other use cases where all tables, functions and
> > other objects need to be replicated,
> > (for example, creating a logical replica for major version upgrade)
> > there won't be any syntax difference to
> > handle and the schemas are supposed to match exactly between the
> > pub/sub. In other words the user seeks to create an identical replica
> > of the source database and the DDLs should be replicated
> > as is in this case.
> >
>
> I think even for database-level replication we can't assume that
> source and target will always have the same data in which case "Create
> Table As ..", "Alter Table .. " kind of statements can't be replicated
> as it is because that can lead to different results. The other point
> is tomorrow we can extend the database level option/syntax to exclude
> a few objects (something like [1]) as well in which case we again need
> to filter at the publisher level.
Good point.
Regarding the idea of using the parse-tree representation produced by
nodeToString(), I’ve not read the patch yet but I'm not sure it's a
good idea. A field name of a node could be changed in a major version.
If a publisher sends a parse-tree string representation to a newer
major version subscriber, the subscriber needs to be able to parse the
old format parse-tree string representation in order to reconstruct
the DDL, which reduces the maintainability much. On the other hand,
the format of deparsed json string would not be changed often.
>
> >
> So I think it's an overkill to use deparser for
> > such use cases. It also costs more space and
> > time using deparsing. For example, the following simple ALTER TABLE
> > command incurs 11 times more space
> > in the WAL record if we were to use the format from the deparser,
> > there will also be time and CPU overhead from the deparser.
> >
> ...
> >
> > So I think it's better to define DDL replication levels [1] to tailor
> > for the two different use cases. We can use different logging format
> > based on the DDL replication level. For example,
> > we can simply log the DDL query string and the search_path for
> > database level DDL replication. But for table level DDL replication we
> > need to use the deparser format in order to
> > handle the potential syntax differences and schema mapping requests.
> >
>
> I think having different logging formats is worth considering but I am
> not sure we can distinguish it for database and table level
> replication because of the reasons mentioned above. One thing which
> may need a different format is the replication of global objects like
> roles, tablespace, etc. but we haven't analyzed them in detail about
> those. I feel we may also need a different syntax altogether to
> replicate such objects. Also, I think we may want to optimize the
> current format in some cases so that the WAL amount could be reduced.
>
> I feel if we think that deparsing is required for this project then
> probably at this stage it would be a good idea to explore ways to have
> independent ways to test it. One way is to do testing via the logical
> replication of DDL (aka via this patch) and the other is to write an
> independent test suite as Sawada-San seems to be speculating above
> [2]. I am not sure if there is any progress yet on the independent
> test suite front yet.
I've attached a WIP patch for adding regression tests for DDL deparse.
The patch can be applied on
v9-0001-Functions-to-deparse-DDL-commands.patch Hou recently
submitted[1]. The basic idea is to define the event trigger to deparse
DDLs, run the regression tests, load the deparsed DDLs to another
database cluster, dump both databases, and compare the dumps. Since
the patch doesn't support deparsing all DDLs and there is a bug[2],
the attached regression test does CREATE TABLE and some ALTER TABLE
instead of running regression tests.
Regards,
[1] https://www.postgresql.org/message-id/OS0PR01MB5716B1526C2EDA66907E733B94B39%40OS0PR01MB5716.jpnprd0...
[2] deparsing "ALTER INDEX tbl_idx ALTER COLUMN 2 SET STATISTICS
1000;" causes an assertion failure.
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
Attachments:
[application/octet-stream] 0001-WIP-Add-regression-tests-for-DDL-deparse.patch (4.9K, ../../CAD21AoBVCoPPRKvU_5-=wEXsa92GsNJFJOcYyXzvoSEJCx5dKw@mail.gmail.com/2-0001-WIP-Add-regression-tests-for-DDL-deparse.patch)
download | inline diff:
From 8847e4502a608597e8bb593e251e990c5a9df99f Mon Sep 17 00:00:00 2001
From: Masahiko Sawada <sawada.mshk@gmail.com>
Date: Wed, 22 Jun 2022 14:29:33 +0900
Subject: [PATCH] WIP: Add regression tests for DDL deparse.
---
src/test/modules/test_ddl_deparse/Makefile | 5 +
.../test_ddl_deparse/t/001_deparse_regress.pl | 135 ++++++++++++++++++
2 files changed, 140 insertions(+)
create mode 100644 src/test/modules/test_ddl_deparse/t/001_deparse_regress.pl
diff --git a/src/test/modules/test_ddl_deparse/Makefile b/src/test/modules/test_ddl_deparse/Makefile
index 3a57a95c84..cf047e2ae2 100644
--- a/src/test/modules/test_ddl_deparse/Makefile
+++ b/src/test/modules/test_ddl_deparse/Makefile
@@ -31,6 +31,11 @@ REGRESS = test_ddl_deparse \
EXTRA_INSTALL = contrib/pg_stat_statements
+REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)
+export REGRESS_SHLIB
+
+TAP_TESTS = 1
+
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
diff --git a/src/test/modules/test_ddl_deparse/t/001_deparse_regress.pl b/src/test/modules/test_ddl_deparse/t/001_deparse_regress.pl
new file mode 100644
index 0000000000..5043b3e02c
--- /dev/null
+++ b/src/test/modules/test_ddl_deparse/t/001_deparse_regress.pl
@@ -0,0 +1,135 @@
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use File::Basename;
+
+my $node1 = PostgreSQL::Test::Cluster->new('main');
+$node1->init;
+
+# Increase some settings that Cluster->new makes too low by default.
+$node1->adjust_conf('postgresql.conf', 'max_connections', '25');
+$node1->append_conf('postgresql.conf',
+ 'max_prepared_transactions = 10');
+
+# Create the event trigger to get deparsed DDLs.
+$node1->start;
+is( $node1->psql('postgres',
+ q(
+begin;
+create table deparsed_ddls(tag text, object_identity text, ddl text);
+
+create or replace function deparse_to_json()
+ returns event_trigger language plpgsql as
+$$
+declare
+ r record;
+begin
+ for r in select * from pg_event_trigger_ddl_commands()
+ loop
+ insert into deparsed_ddls(tag, object_identity, ddl) values (r.command_tag, r.object_identity, pg_catalog.ddl_deparse_to_json(r.command));
+ end loop;
+END;
+$$;
+
+create event trigger ddl_deparse_trig
+on ddl_command_end execute procedure deparse_to_json();
+commit;
+)), 0, 'event trigger created');
+
+my $dlpath = dirname($ENV{REGRESS_SHLIB});
+my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
+
+# Run the regression tests against the main server.
+#
+# FIXME: As the deparsing DDL code is in under development and many DDLs are not
+# supported yet, regression tests againt the main server fails.
+#
+#my $extra_opts = $ENV{EXTRA_REGRESS_OPTS} || "";
+#my $rc =
+# system($ENV{PG_REGRESS}
+# . " $extra_opts "
+# . "--dlpath=\"$dlpath\" "
+# . "--bindir= "
+# . "--host="
+# . $node1->host . " "
+# . "--port="
+# . $node1->port . " "
+# . "--schedule=../../regress/parallel_schedule "
+# . "--use-existing "
+# . "--max-concurrent-tests=20 "
+# . "--inputdir=../../regress "
+# . "--outputdir=\"$outputdir\"");
+#if ($rc != 0)
+#{
+# # Dump out the regression diffs file, if there is one
+# my $diffs = "$outputdir/regression.diffs";
+# if (-e $diffs)
+# {
+# print "=== dumping $diffs ===\n";
+# print slurp_file($diffs);
+# print "=== EOF ===\n";
+# }
+#}
+#
+#is($rc, 0, 'regression tests pass');
+
+# FIXME: temporary tests in lieu of running the regression tests
+$node1->safe_psql(
+ 'postgres',
+ q(
+create table test_tbl (a int, b text, c jsonb);
+alter table test_tbl add column d int;
+alter table test_tbl alter column b type varchar;
+alter table test_tbl alter column a set not null;
+alter table test_tbl drop column d;
+));
+
+# Retrieve the deparsed DDLs.
+my $ddl_sql = '';
+is( $node1->psql(
+ 'postgres',
+ q(select ddl_deparse_expand_command(ddl) || ';' from deparsed_ddls),
+ stdout => \$ddl_sql
+ ), 0, 'dump deparsed DDLs');
+
+# Initialize another database cluster where we load the deparsed DDLs.
+my $node2 = PostgreSQL::Test::Cluster->new('sub');
+$node2->init;
+$node2->start;
+
+# Load the deparsed DDLs.
+$node2->safe_psql('postgres', $ddl_sql);
+
+# Drop the event trigger and the function before taking a logical dump.
+$node1->safe_psql(
+ 'postgres',
+ q(
+drop event trigger ddl_deparse_trig;
+drop function deparse_to_json();
+drop table deparsed_ddls;
+));
+
+# Perform a logical dump of both the main and the sub server, ane check
+# that they match.
+command_ok(
+ [
+ 'pg_dumpall', '-f', $outputdir . '/main.dump',
+ '--no-sync', '-p', $node1->port
+ ],
+ 'dump main server');
+command_ok(
+ [
+ 'pg_dumpall', '-f', $outputdir . '/sub.dump',
+ '--no-sync', '-p', $node2->port
+ ],
+ 'dump main server');
+command_ok(
+ [ 'diff', $outputdir . '/main.dump', $outputdir . '/sub.dump' ],
+ 'compare main and sub dumps');
+
+$node1->stop;
+$node2->stop;
+
+done_testing();
--
2.24.3 (Apple Git-128)
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-22 09:07 vignesh C <vignesh21@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2 siblings, 1 reply; 449+ messages in thread
From: vignesh C @ 2022-06-22 09:07 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Tue, Jun 21, 2022 at 5:49 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Monday, June 20, 2022 11:32 AM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
> >
> > On Saturday, June 18, 2022 3:38 AM Zheng Li <zhengli10@gmail.com> wrote:
> > > On Wed, Jun 15, 2022 at 12:00 AM Amit Kapila <amit.kapila16@gmail.com>
> > > wrote:
> > > >
> > > > On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> > > > >
> > > > >
> > > > > While I agree that the deparser is needed to handle the potential
> > > > > syntax differences between the pub/sub, I think it's only relevant
> > > > > for the use cases where only a subset of tables in the database are
> > > > > replicated. For other use cases where all tables, functions and
> > > > > other objects need to be replicated, (for example, creating a
> > > > > logical replica for major version upgrade) there won't be any syntax
> > > > > difference to handle and the schemas are supposed to match exactly
> > > > > between the pub/sub. In other words the user seeks to create an
> > > > > identical replica of the source database and the DDLs should be
> > > > > replicated as is in this case.
> > > > >
> > > >
> > > > I think even for database-level replication we can't assume that
> > > > source and target will always have the same data in which case "Create
> > > > Table As ..", "Alter Table .. " kind of statements can't be replicated
> > > > as it is because that can lead to different results.
> > > "Create Table As .." is already handled by setting the skipData flag of the
> > > statement parsetreee before replay:
> > >
> > > /*
> > > * Force skipping data population to avoid data inconsistency.
> > > * Data should be replicated from the publisher instead.
> > > */
> > > castmt->into->skipData = true;
> > >
> > > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > > without any syntax change, by enabling the table rewrite replication and
> > > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> > > I've also adopted this approach in my latest patch
> > > 0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch
> > >
> > > > The other point
> > > > is tomorrow we can extend the database level option/syntax to exclude
> > > > a few objects (something like [1]) as well in which case we again need
> > > > to filter at the publisher level
> > >
> > > I think for such cases it's not full database replication and we could treat it as
> > > table level DDL replication, i.e. use the the deparser format.
> >
> > Hi,
> >
> > Here are some points in my mind about the two approaches discussed here.
> >
> > 1) search_patch vs schema qualify
> >
> > Again, I still think it will bring more flexibility and security by schema qualify the
> > objects in DDL command as mentioned before[1].
> >
> > Besides, a schema qualified DDL is also more appropriate for other use
> > cases(e.g. a table-level replication). As it's possible the schema is different
> > between pub/sub and it's easy to cause unexpected and undetectable failure if
> > we just log the search_path.
> >
> > It makes more sense to me to have the same style WAL log(schema qualified)
> > for
> > both database level or table level replication as it will bring more
> > flexibility.
> >
> >
> > > "Create Table As .." is already handled by setting the skipData flag of the
> > > statement parsetreee before replay:
> >
> > 2) About the handling of CREATE TABLE AS:
> >
> > I think it's not a appropriate approach to set the skipdata flag on subscriber
> > as it cannot handle EXECUTE command in CTAS.
> >
> > CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
> >
> > The Prepared statement is a temporary object which we don't replicate. So if
> > you directly execute the original SQL on subscriber, even if you set skipdata
> > it will fail.
> >
> > I think it difficult to make this work as you need handle the create/drop of
> > this prepared statement. And even if we extended subscriber's code to make it
> > work, it doesn't seems like a standard and elegant approach.
> >
> >
> > > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > > without any syntax change, by enabling the table rewrite replication and
> > > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> >
> > 3) About the handling of ALTER TABLE rewrite.
> >
> > The approach I proposed before is based on the event trigger + deparser
> > approach. We were able to improve that approach as we don't need to
> > replicate
> > the rewrite in many cases. For example: we don't need to replicate rewrite dml
> > if there is no volatile/mutable function. We should check and filter these case
> > at publisher (e.g. via deparser) instead of checking that at subscriber.
> >
> > Besides, as discussed, we need to give warning or error for the cases when DDL
> > contains volatile function which would be executed[2]. We should check this at
> > publisher as well(via deparser).
> >
> >
> > > I think for such cases it's not full database replication and we could treat it as
> > > table level DDL replication, i.e. use the the deparser format.
> >
> > 4) I think the point could be that we should make the WAL log format
> > extendable
> > so that we can extend it to support more useful feature(table filter/schema
> > maps/DDL filter). If we just WAL log the original SQL, it seems it's difficult
> > to extend it in the future ?
>
> Attach the new version patch set which added support for CREATE/DROP/ATER
> Sequence and CREATE/DROP Schema ddl commands which are provided by Ajin
> Cherian off list.
>
> The new version patch will also check function's volatility[1] in ALTER TABLE
> command. If any function to be executed is volatile, we report an ERROR.
> Whether WARNING is better to be used here is still under consideration.
>
> [1] https://www.postgresql.org/message-id/CAA4eK1JVynFsj%2BmcRWj9sewR2yNUs6LuNxJ0eN-gNJ83oKcUOQ%40mail.g...
I noticed a couple of things while trying to apply the patch and
review the patch:
1) Creation of temporary table fails infinitely in the subscriber.
CREATE TEMPORARY TABLE temp1 (a int primary key);
The above statement is converted to the below format:
CREATE TEMPORARY TABLE pg_temp.temp1 (a pg_catalog.int4 ,
CONSTRAINT temp1_pkey PRIMARY KEY (a));
While handling the creation of temporary table in the worker, the
worker fails continuously with the following error:
2022-06-22 14:24:01.317 IST [240872] ERROR: schema "pg_temp" does not exist
2022-06-22 14:24:01.317 IST [240872] CONTEXT: processing remote data
for replication origin "pg_16384" during "DDL" in transaction 725
finished at 0/14BBDA8
This error comes from handle_create_table->get_namespace_oid. Here it
checks that the pg_temp namespace is present in the system or not. As
pg_temp namespace is not present it will throw an error.
I saw one issue regarding the partition table mentioned in the commit
message. We should include this in the commit message till this issue
is resolved.
2) There are few whitespace errors while applying patch
git am v9-0001-Functions-to-deparse-DDL-commands.patch
Applying: Functions to deparse DDL commands.
.git/rebase-apply/patch:1480: indent with spaces.
ObjTree *tmp;
.git/rebase-apply/patch:1481: indent with spaces.
char *tmpstr;
.git/rebase-apply/patch:1483: indent with spaces.
tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
.git/rebase-apply/patch:1484: indent with spaces.
tmp = new_objtree_VA("CACHE %{value}s",
.git/rebase-apply/patch:1485: indent with spaces.
2,
warning: squelched 140 whitespace errors
warning: 145 lines add whitespace errors.
Regards,
Vignesh
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-22 22:22 Zheng Li <zhengli10@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-06-22 22:22 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
> Here are some points in my mind about the two approaches discussed here.
>
> 1) search_patch vs schema qualify
>
> Again, I still think it will bring more flexibility and security by schema qualify the
> objects in DDL command as mentioned before[1].
I wonder what security concerns you have? We certainly don't want to
log the search_path
if there are serious security issues.
> Besides, a schema qualified DDL is also more appropriate for other use
> cases(e.g. a table-level replication). As it's possible the schema is different
> between pub/sub and it's easy to cause unexpected and undetectable failure if
> we just log the search_path.
>
> It makes more sense to me to have the same style WAL log(schema qualified) for
> both database level or table level replication as it will bring more
> flexibility.
I think it's reasonable to consider using different formats for the
two different use cases.
Especially if the space and time overhead of the deparser format
sticks out. I also don't
think we need to use the deparser for global objects DDL such as ROLE
statements because
no schema qualification is needed. Also another issue with ROLE
statements is that they
are not captured by event triggers currently.
> > "Create Table As .." is already handled by setting the skipData flag of the
> > statement parsetreee before replay:
>
> 2) About the handling of CREATE TABLE AS:
>
> I think it's not a appropriate approach to set the skipdata flag on subscriber
> as it cannot handle EXECUTE command in CTAS.
>
> CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
>
> The Prepared statement is a temporary object which we don't replicate. So if
> you directly execute the original SQL on subscriber, even if you set skipdata
> it will fail.
>
> I think it difficult to make this work as you need handle the create/drop of
> this prepared statement. And even if we extended subscriber's code to make it
> work, it doesn't seems like a standard and elegant approach.
This is indeed an interesting case, thanks for pointing this out. One
light weight solution
I can think of is to directly deparse the parsetree on the publisher
into a simple CREATE TABLE statement
without the prepared statement and then replicate the simple CREATE
TABLE statement .
This doesn't have to involve the json format though.
> > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > without any syntax change, by enabling the table rewrite replication and
> > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
>
> 3) About the handling of ALTER TABLE rewrite.
>
> The approach I proposed before is based on the event trigger + deparser
> approach. We were able to improve that approach as we don't need to replicate
> the rewrite in many cases. For example: we don't need to replicate rewrite dml
> if there is no volatile/mutable function. We should check and filter these case
> at publisher (e.g. via deparser) instead of checking that at subscriber.
Surely we can make the check about volatile/mutable functions on the
publisher side
as well. It doesn't have to be done via the deparser.
> Besides, as discussed, we need to give warning or error for the cases when DDL
> contains volatile function which would be executed[2]. We should check this at
> publisher as well(via deparser).
Again, I think the check doesn't have to be done via the deparser.
> > I think for such cases it's not full database replication and we could treat it as
> > table level DDL replication, i.e. use the the deparser format.
>
> 4) I think the point could be that we should make the WAL log format extendable
> so that we can extend it to support more useful feature(table filter/schema
> maps/DDL filter). If we just WAL log the original SQL, it seems it's difficult
> to extend it in the future ?
My point is that for full replication/version upgrade use cases we
don't need to worry about extending
it for features such as schema mapping. Because such use cases
naturally want to keep identical schema
structures.
With Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-23 07:09 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-23 07:09 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Thursday, June 23, 2022 6:22 AM Zheng Li <zhengli10@gmail.com> wrote:
Hi,
>
> > Here are some points in my mind about the two approaches discussed here.
> >
> > 1) search_patch vs schema qualify
> >
> > Again, I still think it will bring more flexibility and security by
> > schema qualify the objects in DDL command as mentioned before[1].
>
> I wonder what security concerns you have? We certainly don't want to log the
> search_path if there are serious security issues.
I was thinking the case when the publisher has two schema "s1, s2" while
subscriber only has schema "s2". If we set publisher's search_patch to
's1, s2' and execute CREATE TABLE xxx (); If we replicate the original SQL
with search_path to subcriber, it would silently create the table on
schema s2 instead of reporting an error "schema s1 doesn't exist" which
looks dangerous to me.
>
> > > "Create Table As .." is already handled by setting the skipData flag
> > > of the statement parsetreee before replay:
> >
> > 2) About the handling of CREATE TABLE AS:
> >
> > I think it's not a appropriate approach to set the skipdata flag on
> > subscriber as it cannot handle EXECUTE command in CTAS.
> >
> > CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
> >
> > The Prepared statement is a temporary object which we don't replicate.
> > So if you directly execute the original SQL on subscriber, even if you
> > set skipdata it will fail.
> >
> > I think it difficult to make this work as you need handle the
> > create/drop of this prepared statement. And even if we extended
> > subscriber's code to make it work, it doesn't seems like a standard and
> elegant approach.
>
> This is indeed an interesting case, thanks for pointing this out. One light weight
> solution I can think of is to directly deparse the parsetree on the publisher into
> a simple CREATE TABLE statement without the prepared statement and then
> replicate the simple CREATE TABLE statement .
> This doesn't have to involve the json format though.
I thought about this solution as well. But I am not very sure about this,
I feel it looks a bit hacky to directly do this instead of using a standard
event trigger(Or introduce a new type event trigger).
> > > "Alter Table .. " that rewrites with volatile expressions can also
> > > be handled without any syntax change, by enabling the table rewrite
> > > replication and converting the rewrite inserts to updates. ZJ's patch
> introduced this solution.
> >
> > 3) About the handling of ALTER TABLE rewrite.
> >
> > The approach I proposed before is based on the event trigger +
> > deparser approach. We were able to improve that approach as we don't
> > need to replicate the rewrite in many cases. For example: we don't
> > need to replicate rewrite dml if there is no volatile/mutable
> > function. We should check and filter these case at publisher (e.g. via
> deparser) instead of checking that at subscriber.
>
> Surely we can make the check about volatile/mutable functions on the
> publisher side as well. It doesn't have to be done via the deparser.
>
> > Besides, as discussed, we need to give warning or error for the cases
> > when DDL contains volatile function which would be executed[2]. We
> > should check this at publisher as well(via deparser).
>
> Again, I think the check doesn't have to be done via the deparser.
Personally, I think it's not great to add lots of logical replication
related code(check for rewrite DDL, check for function volatility) in
utility.c or tablecmds.c which seems a bit ad-hoc.
Best regards,
Hou zj
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-23 10:00 Amit Kapila <amit.kapila16@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-23 10:00 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Wed, Jun 22, 2022 at 11:09 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> I've attached a WIP patch for adding regression tests for DDL deparse.
> The patch can be applied on
> v9-0001-Functions-to-deparse-DDL-commands.patch Hou recently
> submitted[1]. The basic idea is to define the event trigger to deparse
> DDLs, run the regression tests, load the deparsed DDLs to another
> database cluster, dump both databases, and compare the dumps.
>
Thanks for working on this. It is a good start. I think this will be
helpful to see the missing DDL support. Do we want to run this as part
of every regression run? Won't it increase the regression time as this
seems to run internally the regression tests twice?
Do we need a different trigger to capture drop cases as there are
separate deparsing routines for them, for example
deparse_drop_table()?
> [2] deparsing "ALTER INDEX tbl_idx ALTER COLUMN 2 SET STATISTICS
> 1000;" causes an assertion failure.
>
Sorry, it is not clear to me whether you are talking about some
pre-existing bug or a bug in the proposed patch?
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-23 12:43 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 0 replies; 449+ messages in thread
From: Alvaro Herrera @ 2022-06-23 12:43 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On 2022-Jun-15, houzj.fnst@fujitsu.com wrote:
> On Wednesday, June 15, 2022 8:14 AM Zheng Li <zhengli10@gmail.com> wrote:
> > How does the deparser deparses CREATE FUNCTION STATEMENT? Will it
> > schema qualify
> > objects inside the function definition?
>
> The current deparser doesn't schema qualify objects inside the function
> source as we won't know the schema of inner objects until the function is
> executed. The deparser will only schema qualify the objects around
> function declaration Like:
>
> CREATE FUNCTION [public].test_func(i [pg_catalog].int4 ) RETURNS [pg_catalog].int4 LANGUAGE plpgsql
Right, this is by design. There is no way to deparse a function body --
as far as the backend is concerned, the body is just an opaque string.
That string is to be interpreted by the language handler only.
I don't know if it's possible to do different for non-core PLs, but I do
not think we have to worry about them in the Postgres implementation.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"La victoria es para quien se atreve a estar solo"
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-24 02:39 Masahiko Sawada <sawada.mshk@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Masahiko Sawada @ 2022-06-24 02:39 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Thu, Jun 23, 2022 at 7:00 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Jun 22, 2022 at 11:09 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> >
> > I've attached a WIP patch for adding regression tests for DDL deparse.
> > The patch can be applied on
> > v9-0001-Functions-to-deparse-DDL-commands.patch Hou recently
> > submitted[1]. The basic idea is to define the event trigger to deparse
> > DDLs, run the regression tests, load the deparsed DDLs to another
> > database cluster, dump both databases, and compare the dumps.
> >
>
> Thanks for working on this. It is a good start. I think this will be
> helpful to see the missing DDL support. Do we want to run this as part
> of every regression run? Won't it increase the regression time as this
> seems to run internally the regression tests twice?
Yes, It will increase the regression test time but we already do a
similar thing in 002_pg_upgrade.pl and 027_stream_regress.pl and it
seems to be worth adding to me.
>
> Do we need a different trigger to capture drop cases as there are
> separate deparsing routines for them, for example
> deparse_drop_table()?
Right, we need to capture drop cases by another trigger.
>
> > [2] deparsing "ALTER INDEX tbl_idx ALTER COLUMN 2 SET STATISTICS
> > 1000;" causes an assertion failure.
> >
>
> Sorry, it is not clear to me whether you are talking about some
> pre-existing bug or a bug in the proposed patch?
I meant there is a bug in the v9 DDL deparse patch.
Regards,
--
Masahiko Sawada
EDB: https://www.enterprisedb.com/
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-24 10:19 Amit Kapila <amit.kapila16@gmail.com>
parent: Masahiko Sawada <sawada.mshk@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-06-24 10:19 UTC (permalink / raw)
To: Masahiko Sawada <sawada.mshk@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
On Fri, Jun 24, 2022 at 8:10 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
>
> On Thu, Jun 23, 2022 at 7:00 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Wed, Jun 22, 2022 at 11:09 AM Masahiko Sawada <sawada.mshk@gmail.com> wrote:
> > >
> > > I've attached a WIP patch for adding regression tests for DDL deparse.
> > > The patch can be applied on
> > > v9-0001-Functions-to-deparse-DDL-commands.patch Hou recently
> > > submitted[1]. The basic idea is to define the event trigger to deparse
> > > DDLs, run the regression tests, load the deparsed DDLs to another
> > > database cluster, dump both databases, and compare the dumps.
> > >
> >
> > Thanks for working on this. It is a good start. I think this will be
> > helpful to see the missing DDL support. Do we want to run this as part
> > of every regression run? Won't it increase the regression time as this
> > seems to run internally the regression tests twice?
>
> Yes, It will increase the regression test time but we already do a
> similar thing in 002_pg_upgrade.pl and 027_stream_regress.pl and it
> seems to be worth adding to me.
>
Fair enough. I think here we need to run it twice once before deparse
and once after deparsing whereas those tests seem to be running it one
time. That might not matter much but we can check the timing
difference. I agree that we anyway need something like this to verify
the deparsing code.
> >
> > Do we need a different trigger to capture drop cases as there are
> > separate deparsing routines for them, for example
> > deparse_drop_table()?
>
> Right, we need to capture drop cases by another trigger.
>
And probably something for alter subcommands as well.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-26 17:50 Alvaro Herrera <alvherre@alvh.no-ip.org>
parent: vignesh C <vignesh21@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Alvaro Herrera @ 2022-06-26 17:50 UTC (permalink / raw)
To: vignesh C <vignesh21@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On 2022-Jun-22, vignesh C wrote:
> 1) Creation of temporary table fails infinitely in the subscriber.
> CREATE TEMPORARY TABLE temp1 (a int primary key);
>
> The above statement is converted to the below format:
> CREATE TEMPORARY TABLE pg_temp.temp1 (a pg_catalog.int4 ,
> CONSTRAINT temp1_pkey PRIMARY KEY (a));
> While handling the creation of temporary table in the worker, the
> worker fails continuously with the following error:
> 2022-06-22 14:24:01.317 IST [240872] ERROR: schema "pg_temp" does not exist
Perhaps one possible fix is to change the JSON format string used in
deparse_CreateStmt. Currently, the following is used:
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
(Note that the word for the "persistence" element here comes straight
from relation->rd_rel->relpersistence.) Maybe it would be more
appropriate to set the schema to empty when the table is temp, since the
temporary-ness is in the %{persistence} element, and thus there is no
need to schema-qualify the table name.
However, that would still replicate a command that involves a temporary
table, which perhaps should not be considered fit for replication. So
another school of thought is that if the %{persistence} is set to
TEMPORARY, then it would be better to skip replicating the command
altogether. I'm not sure how to plug that in the replication layer,
however.
--
Álvaro Herrera PostgreSQL Developer — https://www.EnterpriseDB.com/
"At least to kernel hackers, who really are human, despite occasional
rumors to the contrary" (LWN.net)
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-28 03:26 Amit Kapila <amit.kapila16@gmail.com>
parent: Alvaro Herrera <alvherre@alvh.no-ip.org>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-28 03:26 UTC (permalink / raw)
To: Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: vignesh C <vignesh21@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Sun, Jun 26, 2022 at 11:47 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
>
> On 2022-Jun-22, vignesh C wrote:
>
> > 1) Creation of temporary table fails infinitely in the subscriber.
> > CREATE TEMPORARY TABLE temp1 (a int primary key);
> >
> > The above statement is converted to the below format:
> > CREATE TEMPORARY TABLE pg_temp.temp1 (a pg_catalog.int4 ,
> > CONSTRAINT temp1_pkey PRIMARY KEY (a));
> > While handling the creation of temporary table in the worker, the
> > worker fails continuously with the following error:
> > 2022-06-22 14:24:01.317 IST [240872] ERROR: schema "pg_temp" does not exist
>
> Perhaps one possible fix is to change the JSON format string used in
> deparse_CreateStmt. Currently, the following is used:
>
> + if (node->ofTypename)
> + fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
> + "OF %{of_type}T %{table_elements}s "
> + "%{with_clause}s %{on_commit}s %{tablespace}s";
> + else
> + fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
> + "(%{table_elements:, }s) %{inherits}s "
> + "%{with_clause}s %{on_commit}s %{tablespace}s";
> +
> + createStmt =
> + new_objtree_VA(fmtstr, 1,
> + "persistence", ObjTypeString,
> + get_persistence_str(relation->rd_rel->relpersistence));
>
> (Note that the word for the "persistence" element here comes straight
> from relation->rd_rel->relpersistence.) Maybe it would be more
> appropriate to set the schema to empty when the table is temp, since the
> temporary-ness is in the %{persistence} element, and thus there is no
> need to schema-qualify the table name.
>
>
> However, that would still replicate a command that involves a temporary
> table, which perhaps should not be considered fit for replication. So
> another school of thought is that if the %{persistence} is set to
> TEMPORARY, then it would be better to skip replicating the command
> altogether.
>
+1. I think it doesn't make sense to replicate temporary tables.
Similarly, we don't need to replicate the unlogged tables.
> I'm not sure how to plug that in the replication layer,
> however.
>
I see two possibilities (a) We can check the persistence and skip
logging it in the event trigger where the patch deparses the DDL and
WAL log it, or (b) We can add a similar check in pgoutput.c where we
send the DDL to downstream.
I feel (a) is better unless it is difficult to detect at that stage as
that saves additional WAL.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-28 12:13 Amit Kapila <amit.kapila16@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-28 12:13 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Tue, Jun 21, 2022 at 5:49 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Monday, June 20, 2022 11:32 AM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
> >
>
> Attach the new version patch set which added support for CREATE/DROP/ATER
> Sequence and CREATE/DROP Schema ddl commands which are provided by Ajin
> Cherian off list.
>
Few questions/comments on v9-0001-Functions-to-deparse-DDL-commands
===========================================================
1.
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
...
...
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
The function mentioned in the comments seems to be changed to
format_type_extended in commit a26116c6. If so, change it accordingly.
2. It is not clear to me why format_type_detailed needs to use
'peculiar_typmod' label and then goto statement? In
format_type_extended, we have similar code but without using the goto
statement, so can't we use a similar way here as well?
3.
format_type_detailed()
{
...
+ typeform->typstorage != 'p')
It is better to replace the constant 'p' with TYPSTORAGE_PLAIN.
4. It seems to me that the handling of some of the built-in types is
different between format_type_detailed and format_type_extended. Can
we add some comments to explain the same?
5.
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
{
...
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
...
}
Why do we need to append the objects (tablespace, with clause, etc.)
when they are not present in the actual CREATE TABLE statement? The
reason to ask this is that this makes the string that we want to send
downstream much longer than the actual statement given by the user on
the publisher.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-28 16:47 vignesh C <vignesh21@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2 siblings, 0 replies; 449+ messages in thread
From: vignesh C @ 2022-06-28 16:47 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Tue, Jun 21, 2022 at 5:49 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Monday, June 20, 2022 11:32 AM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
> >
> > On Saturday, June 18, 2022 3:38 AM Zheng Li <zhengli10@gmail.com> wrote:
> > > On Wed, Jun 15, 2022 at 12:00 AM Amit Kapila <amit.kapila16@gmail.com>
> > > wrote:
> > > >
> > > > On Wed, Jun 15, 2022 at 5:44 AM Zheng Li <zhengli10@gmail.com> wrote:
> > > > >
> > > > >
> > > > > While I agree that the deparser is needed to handle the potential
> > > > > syntax differences between the pub/sub, I think it's only relevant
> > > > > for the use cases where only a subset of tables in the database are
> > > > > replicated. For other use cases where all tables, functions and
> > > > > other objects need to be replicated, (for example, creating a
> > > > > logical replica for major version upgrade) there won't be any syntax
> > > > > difference to handle and the schemas are supposed to match exactly
> > > > > between the pub/sub. In other words the user seeks to create an
> > > > > identical replica of the source database and the DDLs should be
> > > > > replicated as is in this case.
> > > > >
> > > >
> > > > I think even for database-level replication we can't assume that
> > > > source and target will always have the same data in which case "Create
> > > > Table As ..", "Alter Table .. " kind of statements can't be replicated
> > > > as it is because that can lead to different results.
> > > "Create Table As .." is already handled by setting the skipData flag of the
> > > statement parsetreee before replay:
> > >
> > > /*
> > > * Force skipping data population to avoid data inconsistency.
> > > * Data should be replicated from the publisher instead.
> > > */
> > > castmt->into->skipData = true;
> > >
> > > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > > without any syntax change, by enabling the table rewrite replication and
> > > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> > > I've also adopted this approach in my latest patch
> > > 0012-Support-replication-of-ALTER-TABLE-commands-that-rew.patch
> > >
> > > > The other point
> > > > is tomorrow we can extend the database level option/syntax to exclude
> > > > a few objects (something like [1]) as well in which case we again need
> > > > to filter at the publisher level
> > >
> > > I think for such cases it's not full database replication and we could treat it as
> > > table level DDL replication, i.e. use the the deparser format.
> >
> > Hi,
> >
> > Here are some points in my mind about the two approaches discussed here.
> >
> > 1) search_patch vs schema qualify
> >
> > Again, I still think it will bring more flexibility and security by schema qualify the
> > objects in DDL command as mentioned before[1].
> >
> > Besides, a schema qualified DDL is also more appropriate for other use
> > cases(e.g. a table-level replication). As it's possible the schema is different
> > between pub/sub and it's easy to cause unexpected and undetectable failure if
> > we just log the search_path.
> >
> > It makes more sense to me to have the same style WAL log(schema qualified)
> > for
> > both database level or table level replication as it will bring more
> > flexibility.
> >
> >
> > > "Create Table As .." is already handled by setting the skipData flag of the
> > > statement parsetreee before replay:
> >
> > 2) About the handling of CREATE TABLE AS:
> >
> > I think it's not a appropriate approach to set the skipdata flag on subscriber
> > as it cannot handle EXECUTE command in CTAS.
> >
> > CREATE TABLE q5_prep_results AS EXECUTE q5(200, 'DTAAAA');
> >
> > The Prepared statement is a temporary object which we don't replicate. So if
> > you directly execute the original SQL on subscriber, even if you set skipdata
> > it will fail.
> >
> > I think it difficult to make this work as you need handle the create/drop of
> > this prepared statement. And even if we extended subscriber's code to make it
> > work, it doesn't seems like a standard and elegant approach.
> >
> >
> > > "Alter Table .. " that rewrites with volatile expressions can also be handled
> > > without any syntax change, by enabling the table rewrite replication and
> > > converting the rewrite inserts to updates. ZJ's patch introduced this solution.
> >
> > 3) About the handling of ALTER TABLE rewrite.
> >
> > The approach I proposed before is based on the event trigger + deparser
> > approach. We were able to improve that approach as we don't need to
> > replicate
> > the rewrite in many cases. For example: we don't need to replicate rewrite dml
> > if there is no volatile/mutable function. We should check and filter these case
> > at publisher (e.g. via deparser) instead of checking that at subscriber.
> >
> > Besides, as discussed, we need to give warning or error for the cases when DDL
> > contains volatile function which would be executed[2]. We should check this at
> > publisher as well(via deparser).
> >
> >
> > > I think for such cases it's not full database replication and we could treat it as
> > > table level DDL replication, i.e. use the the deparser format.
> >
> > 4) I think the point could be that we should make the WAL log format
> > extendable
> > so that we can extend it to support more useful feature(table filter/schema
> > maps/DDL filter). If we just WAL log the original SQL, it seems it's difficult
> > to extend it in the future ?
>
> Attach the new version patch set which added support for CREATE/DROP/ATER
> Sequence and CREATE/DROP Schema ddl commands which are provided by Ajin
> Cherian off list.
>
> The new version patch will also check function's volatility[1] in ALTER TABLE
> command. If any function to be executed is volatile, we report an ERROR.
> Whether WARNING is better to be used here is still under consideration.
Few comments:
1) I found a null pointer access while trying to use the ddl feature:
#0 0x000055e9deb2904b in EventTriggerTableInitWrite
(real_create=0x55e9e0eb0288, address=...) at event_trigger.c:989
#1 0x000055e9deb15745 in create_ctas_internal
(attrList=0x55e9e0eb0140, into=0x55e9e0e86710) at createas.c:154
#2 0x000055e9deb16181 in intorel_startup (self=0x55e9e0e86d00,
operation=1, typeinfo=0x55e9e0ea99d0) at createas.c:526
#3 0x000055e9debdcfdc in standard_ExecutorRun
(queryDesc=0x55e9e0e8f240, direction=ForwardScanDirection, count=0,
execute_once=true) at execMain.c:352
#4 0x000055e9debdcf0b in ExecutorRun (queryDesc=0x55e9e0e8f240,
direction=ForwardScanDirection, count=0, execute_once=true) at
execMain.c:307
#5 0x000055e9deb15cd2 in ExecCreateTableAs (pstate=0x55e9e0e86830,
stmt=0x55e9e0e717e8, params=0x0, queryEnv=0x0, qc=0x7fff45517190) at
createas.c:346
#6 0x000055e9dee3202a in ProcessUtilitySlow (pstate=0x55e9e0e86830,
pstmt=0x55e9e0e70b18,
queryString=0x55e9e0e483e0 "CREATE TEMP TABLE
pg_temp_5.pg_temp_24961_2 AS SELECT mv.ctid AS tid,
newdata.*::pg_temp_5.pg_temp_24961 AS newdata FROM public.sro_index_mv
mv FULL JOIN pg_temp_5.pg_temp_24961 newdata ON (newdata.c "...,
context=PROCESS_UTILITY_QUERY, params=0x0, queryEnv=0x0,
dest=0x55e9df2e7640 <spi_printtupDR>, qc=0x7fff45517190) at
utility.c:1669
#7 0x000055e9dee30b5d in standard_ProcessUtility (pstmt=0x55e9e0e70b18,
queryString=0x55e9e0e483e0 "CREATE TEMP TABLE
pg_temp_5.pg_temp_24961_2 AS SELECT mv.ctid AS tid,
newdata.*::pg_temp_5.pg_temp_24961 AS newdata FROM public.sro_index_mv
mv FULL JOIN pg_temp_5.pg_temp_24961 newdata ON (newdata.c "...,
readOnlyTree=true, context=PROCESS_UTILITY_QUERY, params=0x0,
queryEnv=0x0, dest=0x55e9df2e7640 <spi_printtupDR>, qc=0x7fff45517190)
at utility.c:1074
#8 0x000055e9dee2fac7 in ProcessUtility (pstmt=0x55e9e0e19538,
queryString=0x55e9e0e483e0 "CREATE TEMP TABLE
pg_temp_5.pg_temp_24961_2 AS SELECT mv.ctid AS tid,
newdata.*::pg_temp_5.pg_temp_24961 AS newdata FROM public.sro_index_mv
mv FULL JOIN pg_temp_5.pg_temp_24961 newdata ON (newdata.c "...,
readOnlyTree=true, context=PROCESS_UTILITY_QUERY, params=0x0,
queryEnv=0x0, dest=0x55e9df2e7640 <spi_printtupDR>, qc=0x7fff45517190)
at utility.c:530
#9 0x000055e9dec3c5f1 in _SPI_execute_plan (plan=0x7fff45517230,
options=0x7fff45517200, snapshot=0x0, crosscheck_snapshot=0x0,
fire_triggers=true) at spi.c:2693
#10 0x000055e9dec38ea8 in SPI_execute (
src=0x55e9e0e483e0 "CREATE TEMP TABLE pg_temp_5.pg_temp_24961_2 AS
SELECT mv.ctid AS tid, newdata.*::pg_temp_5.pg_temp_24961 AS newdata
FROM public.sro_index_mv mv FULL JOIN pg_temp_5.pg_temp_24961 newdata
ON (newdata.c "..., read_only=false, tcount=0) at spi.c:618
#11 0x000055e9dec38efd in SPI_exec (
src=0x55e9e0e483e0 "CREATE TEMP TABLE pg_temp_5.pg_temp_24961_2 AS
SELECT mv.ctid AS tid, newdata.*::pg_temp_5.pg_temp_24961 AS newdata
FROM public.sro_index_mv mv FULL JOIN pg_temp_5.pg_temp_24961 newdata
ON (newdata.c "..., tcount=0) at spi.c:630
#12 0x000055e9deb5360a in refresh_by_match_merge (matviewOid=24961,
tempOid=24966, relowner=24909, save_sec_context=0) at matview.c:795
#13 0x000055e9deb528ca in ExecRefreshMatView (stmt=0x55e9e0d3e670,
queryString=0x55e9e0d3dc18 "REFRESH MATERIALIZED VIEW CONCURRENTLY
sro_index_mv;", params=0x0, qc=0x7fff45517d40) at matview.c:317
currentCommand seems to be null here:
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+
EVT_TableInitWrite,
+
"table_init_write",
+
&trigdata);
2) Missing copyright information:
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..4f3c55f
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,12 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
3) This line removal is not required:
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
4) We could add few comments:
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
5) Missing function header:
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
6) This can be removed:
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+
"type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds,
new_object_object(tmp));
+ break;
+*/
Regards,
Vignesh
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-29 03:06 Amit Kapila <amit.kapila16@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-29 03:06 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> 5.
> +static ObjTree *
> +deparse_CreateStmt(Oid objectId, Node *parsetree)
> {
> ...
> + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
> + if (node->tablespacename)
> + append_string_object(tmp, "tablespace", node->tablespacename);
> + else
> + {
> + append_null_object(tmp, "tablespace");
> + append_bool_object(tmp, "present", false);
> + }
> + append_object_object(createStmt, "tablespace", tmp);
> ...
> }
>
> Why do we need to append the objects (tablespace, with clause, etc.)
> when they are not present in the actual CREATE TABLE statement? The
> reason to ask this is that this makes the string that we want to send
> downstream much longer than the actual statement given by the user on
> the publisher.
>
After thinking some more on this, it seems the user may want to
optionally change some of these attributes, for example, on the
subscriber, it may want to associate the table with a different
tablespace. I think to address that we can append these additional
attributes optionally, say via an additional parameter
(append_all_options/append_all_attributes or something like that) in
exposed APIs like deparse_utility_command().
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-29 09:47 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-29 09:47 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; +Cc: vignesh C <vignesh21@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Tuesday, June 28, 2022 11:27 AM Amit Kapila <amit.kapila16@gmail.com>
> On Sun, Jun 26, 2022 at 11:47 PM Alvaro Herrera <alvherre@alvh.no-ip.org>
> wrote:
> >
> > On 2022-Jun-22, vignesh C wrote:
> >
> > > 1) Creation of temporary table fails infinitely in the subscriber.
> > > CREATE TEMPORARY TABLE temp1 (a int primary key);
> > >
> > > The above statement is converted to the below format:
> > > CREATE TEMPORARY TABLE pg_temp.temp1 (a pg_catalog.int4 ,
> > > CONSTRAINT temp1_pkey PRIMARY KEY (a)); While handling the creation
> > > of temporary table in the worker, the worker fails continuously with
> > > the following error:
> > > 2022-06-22 14:24:01.317 IST [240872] ERROR: schema "pg_temp" does
> > > not exist
> >
> > Perhaps one possible fix is to change the JSON format string used in
> > deparse_CreateStmt. Currently, the following is used:
> >
> > + if (node->ofTypename)
> > + fmtstr = "CREATE %{persistence}s
> TABLE %{if_not_exists}s %{identity}D "
> > + "OF %{of_type}T %{table_elements}s "
> > + "%{with_clause}s %{on_commit}s %{tablespace}s";
> > + else
> > + fmtstr = "CREATE %{persistence}s
> TABLE %{if_not_exists}s %{identity}D "
> > + "(%{table_elements:, }s) %{inherits}s "
> > + "%{with_clause}s %{on_commit}s
> > + %{tablespace}s";
> > +
> > + createStmt =
> > + new_objtree_VA(fmtstr, 1,
> > + "persistence", ObjTypeString,
> > +
> > + get_persistence_str(relation->rd_rel->relpersistence));
> >
> > (Note that the word for the "persistence" element here comes straight
> > from relation->rd_rel->relpersistence.) Maybe it would be more
> > appropriate to set the schema to empty when the table is temp, since
> > the temporary-ness is in the %{persistence} element, and thus there is
> > no need to schema-qualify the table name.
> >
> >
> > However, that would still replicate a command that involves a
> > temporary table, which perhaps should not be considered fit for
> > replication. So another school of thought is that if the
> > %{persistence} is set to TEMPORARY, then it would be better to skip
> > replicating the command altogether.
> >
>
> +1. I think it doesn't make sense to replicate temporary tables.
> Similarly, we don't need to replicate the unlogged tables.
I agree that we don’t need to replicate temporary tables.
For unlogged table, one thing I noticed is that we always replicate the
DDL action on unlogged table in streaming replication. So, to be
consistent, maybe we need to generate WAL for DDL on unlogged table as
well ?
Best regards,
Hou zj
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-06-29 09:54 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 2 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-06-29 09:54 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Wednesday, June 29, 2022 11:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila <amit.kapila16@gmail.com>
> wrote:
> >
> > 5.
> > +static ObjTree *
> > +deparse_CreateStmt(Oid objectId, Node *parsetree)
> > {
> > ...
> > + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0); if
> > + (node->tablespacename) append_string_object(tmp, "tablespace",
> > + node->tablespacename); else { append_null_object(tmp, "tablespace");
> > + append_bool_object(tmp, "present", false); }
> > + append_object_object(createStmt, "tablespace", tmp);
> > ...
> > }
> >
> > Why do we need to append the objects (tablespace, with clause, etc.)
> > when they are not present in the actual CREATE TABLE statement? The
> > reason to ask this is that this makes the string that we want to send
> > downstream much longer than the actual statement given by the user on
> > the publisher.
> >
>
> After thinking some more on this, it seems the user may want to optionally
> change some of these attributes, for example, on the subscriber, it may want to
> associate the table with a different tablespace. I think to address that we can
> append these additional attributes optionally, say via an additional parameter
> (append_all_options/append_all_attributes or something like that) in exposed
> APIs like deparse_utility_command().
I agree and will research this part.
And here is the new version patch set.
Most of changes are in the deparser which include:
support CREATE PARTITIONED TABLE
support ALTER ATTACH/DETACH PARTITION
support CREATE/ALTER TABLE with ACCESS METHOD
support CREATE TABLE OF
support CREATE/ALTER TABLE with GENERATED COLUMN
support CREATE/ALTER TABLE with DENTITY COLUMN
support CREATE/ALTER TABLE with COMPRESSION METHOD
support ALTER COLUMN numofcol SET STATISTICS (mentioned by sawada-san [1])
support ALTER SCHEMA
support CRAETE/DROP INDEX
Note that, for ATTACH/DETACH PARTITION, I haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subscriber side is NORMAL table. We will
research this more and improve this later.
Besides, the new version event trigger won't WAL log the DDL whose target
table is a temporary table so that the problem reported by Vignesh[2] is
fixed.
About the recent comment from Amit[3] and Vignesh[4], I will investigate the
comments and address them in next version.
[1] https://www.postgresql.org/message-id/CAD21AoBVCoPPRKvU_5-%3DwEXsa92GsNJFJOcYyXzvoSEJCx5dKw%40mail.g...
[2] https://www.postgresql.org/message-id/CALDaNm33W35pcBE3zOpJhwnYBdBoZDpKxssemAN21NwVhJuong%40mail.gma...
[3] https://www.postgresql.org/message-id/CAA4eK1K88SMoBq%3DDRA4XU-F3FG6qyzCjGMMKsPpcRBPRcrELrw%40mail.g...
[4] https://www.postgresql.org/message-id/CALDaNm3rEA_zmnDMOCT7NqK4aAffhAgooLf8rXjUN%3DYwA8ASFw%40mail.g...
Best regards,
Hou zj
Attachments:
[application/octet-stream] v10-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (14.6K, ../../OS0PR01MB571676298B814F7F5690D92994BB9@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v10-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From 8ef65da8dadcc037e11e4e9356e945ec725c3e90 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Wed, 29 Jun 2022 17:04:32 +0800
Subject: [PATCH] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 18 ++++
src/backend/commands/event_trigger.c | 167 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 221 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 2aadd3a..91e43fd 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -3308,6 +3308,21 @@ removed feature
return alterTableStmt;
}
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
/*
* Handle deparsing of simple commands.
*
@@ -3498,6 +3513,9 @@ deparse_utility_command(CollectedCommand *cmd)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index ddb84d4..7b5c7a0 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -586,7 +588,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -869,6 +872,158 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * TO CHECK: Do we need to generate wal log for unlogged table ? For
+ * now, we still generate wal for unlogged table like what we do for
+ * DDLs for unlogged table in streaming replication.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1149,7 +1304,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1873,6 +2029,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1890,6 +2047,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 12d4b2e..ff75999 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -966,6 +966,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -977,6 +982,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..4d003de 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index 3a9c9f0..467cd65 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bb98c17..5fe28c9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11899,4 +11899,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v10-0001-Functions-to-deparse-DDL-commands.patch (132.6K, ../../OS0PR01MB571676298B814F7F5690D92994BB9@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v10-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From cef50f46f87ec26428be251d4c498fae6f34d17a Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/DROP SCHEMA
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 3545 ++++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 749 ++++++++
src/backend/commands/sequence.c | 34 +
src/backend/commands/tablecmds.c | 17 +-
src/backend/utils/adt/ruleutils.c | 22 +-
src/include/catalog/pg_proc.dat | 7 +-
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 13 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/ruleutils.h | 5 +
11 files changed, 4382 insertions(+), 14 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..2aadd3a
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,3545 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_null_object(void);
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_internal, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Special-case crock for types with strange typmod rules.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ *typarray = false;
+
+peculiar_typmod:
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ /* otherwise, WITH TZ is added by typmod, so fall through */
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ *typname = pstrdup("TIME");
+ break;
+ /* otherwise, WITH TZ is added by typmode, so fall through */
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (array_base_type != InvalidOid &&
+ typeform->typstorage != 'p')
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+
+ /*
+ * If it's an array of one of the types with special typmod rules,
+ * have the element type be processed as above, but now with typarray
+ * set to true.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMETZOID ||
+ type_oid == TIMEOID)
+ goto peculiar_typmod;
+ }
+ else
+ *typarray = false;
+
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Add typmod decoration to the basic type name
+ */
+static char *
+printTypmod(const char *typname, int32 typmod, Oid typmodout)
+{
+ char *res;
+
+ /* Shouldn't be called if typmod is -1 */
+ Assert(typmod >= 0);
+
+ if (typmodout == InvalidOid)
+ {
+ /* Default behavior: just print the integer typmod with parens */
+ res = psprintf("%s(%d)", typname, (int) typmod);
+ }
+ else
+ {
+ /* Use the type-specific typmodout procedure */
+ char *tmstr;
+
+ tmstr = DatumGetCString(OidFunctionCall1(typmodout,
+ Int32GetDatum(typmod)));
+ res = psprintf("%s%s", typname, tmstr);
+ }
+
+ return res;
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter with a NULL value */
+static ObjElem *
+new_null_object(void)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+
+ param->name = NULL;
+ param->objtype = ObjTypeNull;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ ObjElem *param;
+
+ param = new_null_object();
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new boolean parameter */
+static ObjElem *
+new_bool_object(bool value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeBool;
+ param->value.boolean = value;
+
+ return param;
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_bool_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeString;
+ param->value.string = value;
+
+ return param;
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_string_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+static ObjElem *
+new_integer_object(int64 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeInteger;
+ param->value.integer = value;
+
+ return param;
+}
+
+static ObjElem *
+new_float_object(float8 value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeFloat;
+ param->value.flt = value;
+
+ return param;
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeObject;
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object_object(value);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new array parameter */
+static ObjElem *
+new_array_object(List *array)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = NULL;
+ param->objtype = ObjTypeArray;
+ param->value.array = array;
+
+ return param;
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_array_object(array);
+ param->name = name;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else if (isAnyTempNamespace(typnspid))
+ typnsp = pstrdup("pg_temp");
+ else
+ typnsp = get_namespace_name(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *tmp;
+ ObjTree *ident;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+
+ if (alter_table)
+ tmp = new_objtree_VA("%{identity_type}s %{seq_definition: }s", 0);
+ else
+ tmp = new_objtree_VA("%{identity_type}s ( %{seq_definition: }s )", 0);
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(tmp, "present", false);
+ return tmp;
+ }
+
+ if (alter_table)
+ ident = new_objtree_VA("SET GENERATED %{option}s", 0);
+ else
+ ident = new_objtree_VA("GENERATED %{option}s AS IDENTITY", 0);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(ident, "option", "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(ident, "option", "BY DEFAULT");
+ else
+ append_bool_object(ident, "present", false);
+
+ append_object_object(tmp, "identity_type", ident);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(tmp, "seq_definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return tmp;
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{compression}s %{default}s %{not_null}s %{collation}s "
+ "%{generated_column}s %{identity_column}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree_VA("COMPRESSION %{compression_method}I", 0);
+ if (coldef->compression)
+ append_string_object(tmp, "compression_method", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "compression_method");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "compression", tmp);
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "identity_column", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree_VA("GENERATED ALWAYS AS %{generation_expr}s STORED", 0);
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "generation_expr", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "generated_column", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ column = new_objtree_VA("%{name}I WITH OPTIONS %{default}s %{not_null}s", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_InhRelations
+ * Deparse the inherits relations
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+ char *sep;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ sep = "";
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, sep);
+ sep = ", ";
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* if it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+
+ /* all done */
+}
+
+/*
+ * deparse_IndexStmt
+ * deparse an IndexStmt
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ indexStmt =
+ new_objtree_VA("CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I "
+ "ON %{table}D USING %{index_am}s (%{definition}s) "
+ "%{with}s %{tablespace}s %{where_clause}s",
+ 6,
+ "unique", ObjTypeString, node->unique ? "UNIQUE" : "",
+ "concurrently", ObjTypeString,
+ node->concurrent ? "CONCURRENTLY" : "",
+ "if_not_exists", ObjTypeString, node->if_not_exists ? "IF NOT EXISTS" : "",
+ "name", ObjTypeString, RelationGetRelationName(idxrel),
+ "definition", ObjTypeString, definition,
+ "index_am", ObjTypeString, index_am);
+
+ tmp = new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel));
+ append_object_object(indexStmt, "table", tmp);
+
+ /* reloptions */
+ tmp = new_objtree_VA("WITH (%{opts}s)", 0);
+ if (reloptions)
+ append_string_object(tmp, "opts", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "with", tmp);
+
+ /* tablespace */
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}s", 0);
+ if (tablespace)
+ append_string_object(tmp, "tablespace", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "tablespace", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree_VA("WHERE %{where}s", 0);
+ if (whereClause)
+ append_string_object(tmp, "where", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "where_clause", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else if (node->partbound)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "PARTITION OF %{parent_identity}D %{table_elements}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s %{partition_bound}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_string_object(createStmt, "table_kind", "partition");
+
+ append_object_object(createStmt, "parent_identity",
+ elem->value.object);
+ }
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "parents", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "partition_bound",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree_VA("PARTITION BY %{definition}s", 0);
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "definition", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "definition");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "partition_by", tmp);
+
+ /* USING clause */
+ tmp = new_objtree_VA("USING %{access_method}I", 0);
+ if (node->accessMethod)
+ append_string_object(tmp, "access_method", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "access_method");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "access_method", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+ ownerStmt = new_objtree_VA(fmt, 0);
+ append_string_object(ownerStmt, "newowner",
+ get_rolespec_name(node->newowner));
+
+ append_string_object(ownerStmt, "identity",
+ getObjectIdentity(&address, false));
+
+ return ownerStmt;
+}
+
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_SCHEMA:
+ {
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 0);
+ append_string_object(renameStmt, "identity", node->subname);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ append_string_object(renameStmt, "newname", node->newname);
+
+ return renameStmt;
+}
+
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ tmp = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ tmp = new_objtree_VA("RESTART %{value}s",
+ 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+/*
+ * deparse_CreateSchemaStmt
+ * deparse a CreateSchemaStmt
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ createSchema =
+ new_objtree_VA("CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s",
+ 2,
+ "name", ObjTypeString, node->schemaname,
+ "if_not_exists", ObjTypeString,
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+ auth = new_objtree_VA("AUTHORIZATION %{authorization_role}I", 0);
+ if (node->authrole)
+ append_string_object(auth, "authorization_role",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "authorization_role");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "authorization", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/* deparse_CreateSeqStmt
+ * deparse a CreateSeqStmt
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ createSeq =
+ new_objtree_VA("CREATE %{persistence}s SEQUENCE %{identity}D "
+ "%{definition: }s",
+ 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "identity", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * deparse_AlterSeqStmt
+ * deparse an AlterSeqStmt
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq =
+ new_objtree_VA("ALTER SEQUENCE %{identity}D %{definition: }s", 0);
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "identity", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION %{if_not_exists}s",
+ 2, "type", ObjTypeString, "set not null",
+ "if_not_exists", ObjTypeString,
+ subcmd->missing_ok ? "IF EXISTS": "");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3, "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+/*
+removed feature
+ case AT_AddOidsRecurse:
+ case AT_AddOids:
+ tmp = new_objtree_VA("SET WITH OIDS", 1,
+ "type", ObjTypeString, "set with oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+*/
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D %{partition_bound}s",
+ 3, "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId),
+ "partition_bound", ObjTypeString,
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2, "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2, "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2, "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I %{definition}s",
+ 2, "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "definition", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY %{if_not_exists}s",
+ 3, "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name,
+ "if_not_exists", ObjTypeString,
+ subcmd->missing_ok ? "IF EXISTS" : "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_sequence(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SEQUENCE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_schema(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SCHEMA IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_index(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP INDEX IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..95ed18f
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,749 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index ddf219b..ce16206 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1699,6 +1699,40 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ /* open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..ee48470 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5189,11 +5189,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
- context);
+ address = ATExecAttachPartition(wqueue, rel,
+ (PartitionCmd *) cmd->def,
+ context);
else
- ATExecAttachPartitionIdx(wqueue, rel,
- ((PartitionCmd *) cmd->def)->name);
+ address = ATExecAttachPartitionIdx(wqueue, rel,
+ ((PartitionCmd *) cmd->def)->name);
break;
case AT_DetachPartition:
cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode,
@@ -5201,12 +5202,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
Assert(cmd != NULL);
/* ATPrepCmd ensures it must be a table */
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- ATExecDetachPartition(wqueue, tab, rel,
- ((PartitionCmd *) cmd->def)->name,
- ((PartitionCmd *) cmd->def)->concurrent);
+ address = ATExecDetachPartition(wqueue, tab, rel,
+ ((PartitionCmd *) cmd->def)->name,
+ ((PartitionCmd *) cmd->def)->concurrent);
break;
case AT_DetachPartitionFinalize:
- ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
+ address = ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index c3937a6..450c0b7 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -487,8 +487,6 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
@@ -502,7 +500,6 @@ static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1890,6 +1887,12 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2143,6 +2146,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -12020,7 +12032,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12801,7 +12813,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 87aa571..8aa636c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11884,5 +11884,10 @@
proname => 'brin_minmax_multi_summary_send', provolatile => 's',
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
-
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 9da2300..2875106 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..fb43d23
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,13 @@
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_sequence(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_schema(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_index(const char *objidentity, DropBehavior behavior);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..442d1ed 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -26,9 +26,11 @@ extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -41,6 +43,9 @@ extern List *select_rtable_names_for_explain(List *rtable,
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
--
2.7.2.windows.1
[application/octet-stream] v10-0002-Support-DDL-replication.patch (125.4K, ../../OS0PR01MB571676298B814F7F5690D92994BB9@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v10-0002-Support-DDL-replication.patch)
download | inline diff:
From 4113495a73bd0c9d1e8182895fe53dbfcca65e3b Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 21 Jun 2022 17:41:27 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 221 +++++++++++++-
src/backend/commands/publicationcmds.c | 100 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
33 files changed, 1592 insertions(+), 217 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index c365de3..989c49a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 4642527..ddb84d4 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1540,6 +1543,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1573,7 +1577,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1593,6 +1597,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2180,3 +2185,217 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * TO CHECK: Do we need to generate wal log for unlogged table ? For
+ * now, we still generate wal for unlogged table like what we do for
+ * DDLs for unlogged table in streaming replication.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * TO CHECK: Do we need to generate wal log for unlogged table ? For
+ * now, we still generate wal for unlogged table like what we do for
+ * DDLs for unlogged table in streaming replication.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * TO CHECK: Do we need to generate wal log for unlogged table ?
+ * For now, we still generate wal for unlogged table like what we
+ * do for DDLs for unlogged table in streaming replication.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ command = deparse_drop_table(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_TableDropEnd,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "sequence", 8) == 0)
+ {
+ command = deparse_drop_sequence(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "schema", 6) == 0)
+ {
+ command = deparse_drop_schema(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "index", 6) == 0)
+ {
+ command = deparse_drop_index(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..12d4b2e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,49 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1119,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ee48470..ddcce3f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5219,7 +5219,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 38e3b1c..2c667bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 8deae57..331949c 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1982,7 +2156,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2040,6 +2215,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2153,6 +2329,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2338,6 +2515,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0e8fda9..27de475 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7cc9c72..030d491 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3867,6 +3867,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3882,23 +3883,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3914,6 +3921,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3937,6 +3945,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index d1ae699..7171768 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6050,7 +6050,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6085,6 +6085,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6172,6 +6176,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6188,6 +6193,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6201,6 +6207,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6249,6 +6258,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6263,6 +6274,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6273,6 +6286,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 8aa636c..bb98c17 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11890,4 +11890,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..de58035 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v10-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB571676298B814F7F5690D92994BB9@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v10-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From b25920a8213e38de9eb8bb096bdc29d64f3e9482 Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-29 10:48 Amit Kapila <amit.kapila16@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-29 10:48 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Wed, Jun 29, 2022 at 3:24 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Wednesday, June 29, 2022 11:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > >
> > > 5.
> > > +static ObjTree *
> > > +deparse_CreateStmt(Oid objectId, Node *parsetree)
> > > {
> > > ...
> > > + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0); if
> > > + (node->tablespacename) append_string_object(tmp, "tablespace",
> > > + node->tablespacename); else { append_null_object(tmp, "tablespace");
> > > + append_bool_object(tmp, "present", false); }
> > > + append_object_object(createStmt, "tablespace", tmp);
> > > ...
> > > }
> > >
> > > Why do we need to append the objects (tablespace, with clause, etc.)
> > > when they are not present in the actual CREATE TABLE statement? The
> > > reason to ask this is that this makes the string that we want to send
> > > downstream much longer than the actual statement given by the user on
> > > the publisher.
> > >
> >
> > After thinking some more on this, it seems the user may want to optionally
> > change some of these attributes, for example, on the subscriber, it may want to
> > associate the table with a different tablespace. I think to address that we can
> > append these additional attributes optionally, say via an additional parameter
> > (append_all_options/append_all_attributes or something like that) in exposed
> > APIs like deparse_utility_command().
>
> I agree and will research this part.
>
Okay, note that similar handling would be required at other places
like deparse_ColumnDef. Few other comments on
v9-0001-Functions-to-deparse-DDL-commands.
1.
+static ObjElem *new_bool_object(bool value);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static ObjElem *new_array_object(List *array);
+static ObjElem *new_integer_object(int64 value);
+static ObjElem *new_float_object(float8 value);
Here, new_object_object() seems to be declared out-of-order (not in
sync with its order of definition). Similarly, see all other append_*
functions.
2. The function printTypmod in ddl_deparse.c appears to be the same as
the function with the same name in format_type.c. If so, isn't it
better to have a single copy of that function?
3. As I pointed out yesterday, there is some similarity between
format_type_extended and format_type_detailed. Can we try to extract
common functionality? If this is feasible, it is better to do this as
a separate patch. Also, this can obviate the need to expose
printTypmod from format_type.c. I am not really sure if this will be
better than the current one or not but it seems worth trying.
4.
new_objtree_VA()
{
...
switch (type)
+ {
+ case ObjTypeBool:
+ elem = new_bool_object(va_arg(args, int));
+ break;
+ case ObjTypeString:
+ elem = new_string_object(va_arg(args, char *));
+ break;
+ case ObjTypeObject:
+ elem = new_object_object(va_arg(args, ObjTree *));
+ break;
+ case ObjTypeArray:
+ elem = new_array_object(va_arg(args, List *));
+ break;
+ case ObjTypeInteger:
+ elem = new_integer_object(va_arg(args, int64));
+ break;
+ case ObjTypeFloat:
+ elem = new_float_object(va_arg(args, double));
+ break;
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ elem = new_null_object();
...
I feel here ObjType's should be handled in the same order as they are
defined in ObjType.
5. There appears to be common code among node_*_object() functions.
Can we just have one function instead new_object(ObjType, void *val)?
In the function based on type, we can typecast the value. Is there a
reason why that won't be better than current one?
6.
deparse_ColumnDef()
{
...
/* Composite types use a slightly simpler format string */
+ if (composite)
+ column = new_objtree_VA("%{name}I %{coltype}T %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+ else
+ column = new_objtree_VA("%{name}I %{coltype}T %{default}s
%{not_null}s %{collation}s",
+ 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
...
}
To avoid using the same arguments, we can define fmtstr for composite
and non-composite types as the patch is doing in deparse_CreateStmt().
7. It is not clear from comments or otherwise what should be
considered for default format string in functions like
deparse_ColumnDef() or deparse_CreateStmt().
8.
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
I think we need to handle default values for this FIXME.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-30 06:14 Amit Kapila <amit.kapila16@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-06-30 06:14 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Wed, Jun 29, 2022 at 3:17 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Tuesday, June 28, 2022 11:27 AM Amit Kapila <amit.kapila16@gmail.com>
> > On Sun, Jun 26, 2022 at 11:47 PM Alvaro Herrera <alvherre@alvh.no-ip.org>
> > wrote:
> > >
> > > However, that would still replicate a command that involves a
> > > temporary table, which perhaps should not be considered fit for
> > > replication. So another school of thought is that if the
> > > %{persistence} is set to TEMPORARY, then it would be better to skip
> > > replicating the command altogether.
> > >
> >
> > +1. I think it doesn't make sense to replicate temporary tables.
> > Similarly, we don't need to replicate the unlogged tables.
>
> I agree that we don’t need to replicate temporary tables.
>
> For unlogged table, one thing I noticed is that we always replicate the
> DDL action on unlogged table in streaming replication. So, to be
> consistent, maybe we need to generate WAL for DDL on unlogged table as
> well ?
>
We don't seem to WAL log the main fork, so that shouldn't be created
in physical replication whereas in your case it will create the main
fork unless you are doing some special handling for subscribers/apply
worker. We are also not allowed to read the unlogged tables on standby
whereas after logical replication users will be allowed to operate on
them. I think because we need to insert catalog entries for 'create
unlogged table' which can't be selectively logged, it gets replicated
to a physical stand by but I don't see why we need to behave similarly
for logical replication. Can you think of some reason why we need to
be consistent here or in other words why we should replicate DDL for
unlogged tables in logical replication? I am not against it but can't
see the reason for doing it based on the theory that when we are not
going to replicate the data of such tables why should we replicate its
schema.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-06-30 08:16 Amit Kapila <amit.kapila16@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-06-30 08:16 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>; vignesh C <vignesh21@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Thu, Jun 30, 2022 at 11:44 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Jun 29, 2022 at 3:17 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > On Tuesday, June 28, 2022 11:27 AM Amit Kapila <amit.kapila16@gmail.com>
> > >
> > > +1. I think it doesn't make sense to replicate temporary tables.
> > > Similarly, we don't need to replicate the unlogged tables.
> >
> > I agree that we don’t need to replicate temporary tables.
> >
> > For unlogged table, one thing I noticed is that we always replicate the
> > DDL action on unlogged table in streaming replication. So, to be
> > consistent, maybe we need to generate WAL for DDL on unlogged table as
> > well ?
> >
>
> We don't seem to WAL log the main fork, so that shouldn't be created
> in physical replication whereas in your case it will create the main
> fork unless you are doing some special handling for subscribers/apply
> worker. We are also not allowed to read the unlogged tables on standby
> whereas after logical replication users will be allowed to operate on
> them. I think because we need to insert catalog entries for 'create
> unlogged table' which can't be selectively logged, it gets replicated
> to a physical stand by but I don't see why we need to behave similarly
> for logical replication. Can you think of some reason why we need to
> be consistent here or in other words why we should replicate DDL for
> unlogged tables in logical replication?
>
If we don't replicate the unlogged table DDL 'Create Unlogged Table
...', then later if the user changes the table to logged 'Alter Table
... Set Logged' then we need to do some special hacking to create the
table. So, I think we should replicate 'Create Unlogged Table ...'
DDL.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-01 16:52 vignesh C <vignesh21@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: vignesh C @ 2022-07-01 16:52 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Wed, Jun 29, 2022 at 3:25 PM houzj.fnst@fujitsu.com
<houzj.fnst@fujitsu.com> wrote:
>
> On Wednesday, June 29, 2022 11:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila <amit.kapila16@gmail.com>
> > wrote:
> > >
> > > 5.
> > > +static ObjTree *
> > > +deparse_CreateStmt(Oid objectId, Node *parsetree)
> > > {
> > > ...
> > > + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0); if
> > > + (node->tablespacename) append_string_object(tmp, "tablespace",
> > > + node->tablespacename); else { append_null_object(tmp, "tablespace");
> > > + append_bool_object(tmp, "present", false); }
> > > + append_object_object(createStmt, "tablespace", tmp);
> > > ...
> > > }
> > >
> > > Why do we need to append the objects (tablespace, with clause, etc.)
> > > when they are not present in the actual CREATE TABLE statement? The
> > > reason to ask this is that this makes the string that we want to send
> > > downstream much longer than the actual statement given by the user on
> > > the publisher.
> > >
> >
> > After thinking some more on this, it seems the user may want to optionally
> > change some of these attributes, for example, on the subscriber, it may want to
> > associate the table with a different tablespace. I think to address that we can
> > append these additional attributes optionally, say via an additional parameter
> > (append_all_options/append_all_attributes or something like that) in exposed
> > APIs like deparse_utility_command().
>
> I agree and will research this part.
>
> And here is the new version patch set.
> Most of changes are in the deparser which include:
>
> support CREATE PARTITIONED TABLE
> support ALTER ATTACH/DETACH PARTITION
> support CREATE/ALTER TABLE with ACCESS METHOD
> support CREATE TABLE OF
> support CREATE/ALTER TABLE with GENERATED COLUMN
> support CREATE/ALTER TABLE with DENTITY COLUMN
> support CREATE/ALTER TABLE with COMPRESSION METHOD
> support ALTER COLUMN numofcol SET STATISTICS (mentioned by sawada-san [1])
> support ALTER SCHEMA
> support CRAETE/DROP INDEX
>
> Note that, for ATTACH/DETACH PARTITION, I haven't added extra logic on
> subscriber to handle the case where the table on publisher is a PARTITIONED
> TABLE while the target table on subscriber side is NORMAL table. We will
> research this more and improve this later.
>
> Besides, the new version event trigger won't WAL log the DDL whose target
> table is a temporary table so that the problem reported by Vignesh[2] is
> fixed.
>
> About the recent comment from Amit[3] and Vignesh[4], I will investigate the
> comments and address them in next version.
>
> [1] https://www.postgresql.org/message-id/CAD21AoBVCoPPRKvU_5-%3DwEXsa92GsNJFJOcYyXzvoSEJCx5dKw%40mail.g...
> [2] https://www.postgresql.org/message-id/CALDaNm33W35pcBE3zOpJhwnYBdBoZDpKxssemAN21NwVhJuong%40mail.gma...
> [3] https://www.postgresql.org/message-id/CAA4eK1K88SMoBq%3DDRA4XU-F3FG6qyzCjGMMKsPpcRBPRcrELrw%40mail.g...
> [4] https://www.postgresql.org/message-id/CALDaNm3rEA_zmnDMOCT7NqK4aAffhAgooLf8rXjUN%3DYwA8ASFw%40mail.g...
Thanks for the updated patch.
Few comments on 0002 patch:
1) When we create a subscription for a publication with the existing
default PUBLISH parameter having default value as
'insert,update,delete,truncate', we do an initial table sync to get
the initial table data from the publisher to the subscriber. But in
case of a publication created with 'ddl', the subscription expects the
existing initial tables present in the publisher to be created
beforehand in the subscriber. Should this be the default behavior?
Should we do a ddl dump for all the tables and restore the ddl to the
subscription while creating the subscription? Or is this planned as an
option for the later version. If we could do this as part of ddl
logical replication, it will help in reducing the steps further for
logical replication setup. If this will not be supported in this patch
series, probably we could document the behavior and add comments for
this at an appropriate place.
2) When a publication is created with ddl enabled, event triggers
will be created implicitly. Currently these event triggers are also
getting dumped. These should not be dumped. Currently while the
publication is restored, the event trigger will be created and also
the explicit event triggers which were dumped will also get created
resulting in multiple event triggers. The event trigger should not be
dumped.
@@ -4016,6 +4026,15 @@ dumpPublication(Archive *fout, const
PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
3) SYNTAX Support:
Currently creation of "FOR TABLE" publication with ddl is supported.
Should we allow support of ddl for "FOR TABLE" publication. The
creation of the subscription will fail saying the table does not exist
while creating the subscription. Users will have to create all the
tables before creating the subscription. Is this syntax supported for
the use case where the table is altered after the subscription is
created.
ex: create publication pub3 for table t1 with ( PUBLISH= ddl);
4) Few includes can be removed:
4.a) This change is not required, it compiles ok for me without this:
diff --git a/src/backend/access/transam/rmgr.c
b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
4.b) This change is not required, it compiles ok for me without this change:
diff --git a/src/backend/utils/adt/ri_triggers.c
b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
5) The changes are not required for the patch, it can be removed:
5.a) This empty line can be removed
@@ -2153,6 +2329,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |=
pub->pubactions.pubupdate;
entry->pubactions.pubdelete |=
pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |=
pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |=
pub->pubactions.pubddl;
/*
* We want to publish the changes as
the top-most ancestor
@@ -2338,6 +2515,7 @@ rel_sync_cache_publication_cb(Datum arg, int
cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
5.b) This change is done by mistake, this change can be removed:
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb,
ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix =
MemoryContextAlloc(rb->context,
-
prefix_size);
+ change->data.msg.prefix =
MemoryContextAlloc(rb->context, prefix_size);
5.c) This change is done by mistake, this change can be removed:
diff --git a/src/backend/replication/logical/proto.c
b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional,
const char *prefix, Size sz,
- const char *message)
+ bool transactional,
const char *prefix,
+ Size sz, const char *message)
Regards,
Vignesh
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-02 03:21 Amit Kapila <amit.kapila16@gmail.com>
parent: vignesh C <vignesh21@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-07-02 03:21 UTC (permalink / raw)
To: vignesh C <vignesh21@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Fri, Jul 1, 2022 at 10:22 PM vignesh C <vignesh21@gmail.com> wrote:
>
> On Wed, Jun 29, 2022 at 3:25 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
>
> Thanks for the updated patch.
> Few comments on 0002 patch:
> 1) When we create a subscription for a publication with the existing
> default PUBLISH parameter having default value as
> 'insert,update,delete,truncate', we do an initial table sync to get
> the initial table data from the publisher to the subscriber. But in
> case of a publication created with 'ddl', the subscription expects the
> existing initial tables present in the publisher to be created
> beforehand in the subscriber. Should this be the default behavior?
> Should we do a ddl dump for all the tables and restore the ddl to the
> subscription while creating the subscription? Or is this planned as an
> option for the later version.
>
The idea is to develop initial sync (for ddl replication) as a
separate patch. But both need to be integrated at some point.
>
> 3) SYNTAX Support:
> Currently creation of "FOR TABLE" publication with ddl is supported.
> Should we allow support of ddl for "FOR TABLE" publication.
>
The above comment is unclear to me. It seems to me in the first
sentence, you are saying that the "FOR TABLE" syntax is supported and
in the second sentence, you are asking to allow support of it? I think
at this stage, the focus is to build the core part of the feature
(allow ddl replication and deparsing support), and then we can discuss
more on Syntax. Having said that, it will be good if we can support
table-level DDL replication as well in the patch as you seem to be
suggesting.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-02 14:24 vignesh C <vignesh21@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: vignesh C @ 2022-07-02 14:24 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Sat, Jul 2, 2022 at 8:51 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Fri, Jul 1, 2022 at 10:22 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> > On Wed, Jun 29, 2022 at 3:25 PM houzj.fnst@fujitsu.com
> > <houzj.fnst@fujitsu.com> wrote:
> > >
> >
> > Thanks for the updated patch.
> > Few comments on 0002 patch:
> > 1) When we create a subscription for a publication with the existing
> > default PUBLISH parameter having default value as
> > 'insert,update,delete,truncate', we do an initial table sync to get
> > the initial table data from the publisher to the subscriber. But in
> > case of a publication created with 'ddl', the subscription expects the
> > existing initial tables present in the publisher to be created
> > beforehand in the subscriber. Should this be the default behavior?
> > Should we do a ddl dump for all the tables and restore the ddl to the
> > subscription while creating the subscription? Or is this planned as an
> > option for the later version.
> >
>
> The idea is to develop initial sync (for ddl replication) as a
> separate patch. But both need to be integrated at some point.
Yes, that approach makes sense.
> >
> > 3) SYNTAX Support:
> > Currently creation of "FOR TABLE" publication with ddl is supported.
> > Should we allow support of ddl for "FOR TABLE" publication.
> >
>
> The above comment is unclear to me. It seems to me in the first
> sentence, you are saying that the "FOR TABLE" syntax is supported and
> in the second sentence, you are asking to allow support of it? I think
> at this stage, the focus is to build the core part of the feature
> (allow ddl replication and deparsing support), and then we can discuss
> more on Syntax. Having said that, it will be good if we can support
> table-level DDL replication as well in the patch as you seem to be
> suggesting.
I initially thought that supporting "FOR TABLE" publication for ddl
might not be useful as currently the create subscription fails with
table does not exist error. Now that the initial sync for ddl
replication will also be implemented as mentioned in [1], this issue
will be handled. I agree with supporting table-level DDL replication.
[1] - https://www.postgresql.org/message-id/CAA4eK1%2B5zJAT_RYOAEOq8M33s196kR5sDyLQLUXd8Rnqr%2BiB0Q%40mail...
Regards,
Vignesh
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-07-05 08:28 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-07-05 08:28 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Wednesday, June 29, 2022 6:48 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> On Wed, Jun 29, 2022 at 3:24 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> >
> > On Wednesday, June 29, 2022 11:07 AM Amit Kapila
> <amit.kapila16@gmail.com> wrote:
> > >
> > > On Tue, Jun 28, 2022 at 5:43 PM Amit Kapila
> > > <amit.kapila16@gmail.com>
> > > wrote:
> > > >
> > > > 5.
> > > > +static ObjTree *
> > > > +deparse_CreateStmt(Oid objectId, Node *parsetree)
> > > > {
> > > > ...
> > > > + tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0); if
> > > > + (node->tablespacename) append_string_object(tmp, "tablespace",
> > > > + node->tablespacename); else { append_null_object(tmp,
> > > > + node->"tablespace");
> > > > + append_bool_object(tmp, "present", false); }
> > > > + append_object_object(createStmt, "tablespace", tmp);
> > > > ...
> > > > }
> > > >
> > > > Why do we need to append the objects (tablespace, with clause,
> > > > etc.) when they are not present in the actual CREATE TABLE
> > > > statement? The reason to ask this is that this makes the string
> > > > that we want to send downstream much longer than the actual
> > > > statement given by the user on the publisher.
> > > >
> > >
> > > After thinking some more on this, it seems the user may want to
> > > optionally change some of these attributes, for example, on the
> > > subscriber, it may want to associate the table with a different
> > > tablespace. I think to address that we can append these additional
> > > attributes optionally, say via an additional parameter
> > > (append_all_options/append_all_attributes or something like that) in
> exposed APIs like deparse_utility_command().
> >
> > I agree and will research this part.
> >
>
> Okay, note that similar handling would be required at other places like
> deparse_ColumnDef. Few other comments on
> v9-0001-Functions-to-deparse-DDL-commands.
>
> 1.
> +static ObjElem *new_bool_object(bool value); static ObjElem
> +*new_string_object(char *value); static ObjElem
> +*new_object_object(ObjTree *value); static ObjElem
> +*new_array_object(List *array); static ObjElem
> +*new_integer_object(int64 value); static ObjElem
> +*new_float_object(float8 value);
>
> Here, new_object_object() seems to be declared out-of-order (not in sync with
> its order of definition). Similarly, see all other append_* functions.
Changed.
> 2. The function printTypmod in ddl_deparse.c appears to be the same as the
> function with the same name in format_type.c. If so, isn't it better to have a
> single copy of that function?
Changed.
> 3. As I pointed out yesterday, there is some similarity between
> format_type_extended and format_type_detailed. Can we try to extract
> common functionality? If this is feasible, it is better to do this as a separate
> patch. Also, this can obviate the need to expose printTypmod from
> format_type.c. I am not really sure if this will be better than the current one or
> not but it seems worth trying.
It seems the main logic in the two functions is a bit different. For
format_type_detailed, we always try to schema qualify both the user-defined and
built-in type except for some special type(time ..) which we cannot schema
qualify. But in format_type_extended, we don't try to schema qualify the
built-in type. But I will research a bit more about this.
> 4.
> new_objtree_VA()
> {
> ...
> switch (type)
> + {
> + case ObjTypeBool:
> + elem = new_bool_object(va_arg(args, int)); break; case ObjTypeString:
> + elem = new_string_object(va_arg(args, char *)); break; case
> + ObjTypeObject:
> + elem = new_object_object(va_arg(args, ObjTree *)); break; case
> + ObjTypeArray:
> + elem = new_array_object(va_arg(args, List *)); break; case
> + ObjTypeInteger:
> + elem = new_integer_object(va_arg(args, int64)); break; case
> + ObjTypeFloat:
> + elem = new_float_object(va_arg(args, double)); break; case
> + ObjTypeNull:
> + /* Null params don't have a value (obviously) */ elem =
> + new_null_object();
> ...
>
> I feel here ObjType's should be handled in the same order as they are defined in
> ObjType.
Changed.
> 5. There appears to be common code among node_*_object() functions.
> Can we just have one function instead new_object(ObjType, void *val)?
> In the function based on type, we can typecast the value. Is there a reason why
> that won't be better than current one?
I tried to extract some common code into a new function new_object(ObjType,
char *name). I didn't use 'void *' as user could pass a wrong type of value which
we cannot detect which seems a little unsafe.
> 6.
> deparse_ColumnDef()
> {
> ...
> /* Composite types use a slightly simpler format string */
> + if (composite)
> + column = new_objtree_VA("%{name}I %{coltype}T %{collation}s", 3,
> + "type", ObjTypeString, "column", "name", ObjTypeString,
> + coldef->colname, "coltype", ObjTypeObject, new_objtree_for_type(typid,
> + typmod)); else column = new_objtree_VA("%{name}I %{coltype}T
> + %{default}s
> %{not_null}s %{collation}s",
> + 3,
> + "type", ObjTypeString, "column",
> + "name", ObjTypeString, coldef->colname, "coltype", ObjTypeObject,
> + new_objtree_for_type(typid, typmod));
> ...
> }
>
> To avoid using the same arguments, we can define fmtstr for composite and
> non-composite types as the patch is doing in deparse_CreateStmt().
Changed.
> 7. It is not clear from comments or otherwise what should be considered for
> default format string in functions like
> deparse_ColumnDef() or deparse_CreateStmt().
I think it was intended to put most of the modifiable part in default format
string so that user can change what they want using json function. But I think
we could improve, maybe by passing a parameter which decide whether to keep
these keywords in default format string. I will research this.
> 8.
> + * FIXME --- actually, what about default values?
> + */
> +static ObjTree *
> +deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef
> +*coldef)
>
> I think we need to handle default values for this FIXME.
Changed.
Attach the new version patch which address the above
comments and comments from [1][2].
I haven't addressed the latest comments from Vignesh[3],
I will investigate it.
[1] https://www.postgresql.org/message-id/CAA4eK1K88SMoBq%3DDRA4XU-F3FG6qyzCjGMMKsPpcRBPRcrELrw%40mail.g...
[2] https://www.postgresql.org/message-id/CALDaNm3rEA_zmnDMOCT7NqK4aAffhAgooLf8rXjUN%3DYwA8ASFw%40mail.g...
[3] https://www.postgresql.org/message-id/CALDaNm2nFPMxUo%3D0zRUUA-v3_eRwRY%2Bii5nnG_PU%2B6jT7ta9dA%40ma...
Best regards,
Hou zj
Attachments:
[application/octet-stream] v11-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB5716D3B79A607B547F2AB90D94819@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v11-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From b25920a8213e38de9eb8bb096bdc29d64f3e9482 Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
1.8.3.1
[application/octet-stream] v11-0001-Functions-to-deparse-DDL-commands.patch (131.9K, ../../OS0PR01MB5716D3B79A607B547F2AB90D94819@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v11-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From e79790f4eacba771ea6333d67f52cbd827e3f3e1 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides base for logical replication of DDL statements. Currently,
this provides support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/ALTER/DROP SCHEMA
CREATE/DROP INDEX
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 3413 +++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 763 ++++++++
src/backend/commands/sequence.c | 34 +
src/backend/commands/tablecmds.c | 17 +-
src/backend/utils/adt/format_type.c | 3 +-
src/backend/utils/adt/ruleutils.c | 22 +-
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 24 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/builtins.h | 1 +
src/include/utils/ruleutils.h | 5 +
13 files changed, 4277 insertions(+), 15 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..b6b54f7
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,3413 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+static ObjElem *new_object(ObjType type, char *name);
+static ObjElem *new_object_object(ObjTree *value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_extended, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
+
+ /*
+ * Special-case crock for types with strange typmod rules where we put
+ * typmod at the middle of name(e.g. TIME(6) with time zone ). We cannot
+ * schema-qualify nor add quotes to the type name in these cases.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmod. */
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
+ break;
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(void)
+{
+ ObjTree *params;
+
+ params = palloc(sizeof(ObjTree));
+ params->numParams = 0;
+ slist_init(¶ms->params);
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree();
+ append_string_object(tree, "fmt", fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+ elem = new_object(type, NULL);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ break;
+ case ObjTypeBool:
+ elem->value.boolean = va_arg(args, int);
+ break;
+ case ObjTypeString:
+ elem->value.string = va_arg(args, char *);
+ break;
+ case ObjTypeArray:
+ elem->value.array = va_arg(args, List *);
+ break;
+ case ObjTypeInteger:
+ elem->value.integer = va_arg(args, int64);
+ break;
+ case ObjTypeFloat:
+ elem->value.flt = va_arg(args, double);
+ break;
+ case ObjTypeObject:
+ elem->value.object = va_arg(args, ObjTree *);
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/* Allocate a new parameter */
+static ObjElem *
+new_object(ObjType type, char *name)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = name;
+ param->objtype = type;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *name)
+{
+ append_premade_object(tree, new_object(ObjTypeNull, name));
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *name, bool value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeBool, name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
+
+/*
+ * Append a string parameter to a tree.
+ */
+static void
+append_string_object(ObjTree *tree, char *name, char *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object(ObjTypeString, name);
+ param->value.string = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeObject, NULL);
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *name, ObjTree *value)
+{
+ ObjElem *param;
+
+ Assert(name);
+ param = new_object(ObjTypeObject, name);
+ param->value.object = value;
+ append_premade_object(tree, param);
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *name, List *array)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeArray, name);
+ param->value.array = array;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else
+ typnsp = get_namespace_name_or_temp(typnspid);
+
+ /* We don't use new_objtree_VA here because types don't have a "fmt" */
+ typeParam = new_objtree();
+ append_string_object(typeParam, "schemaname", typnsp);
+ append_string_object(typeParam, "typename", typename);
+ append_string_object(typeParam, "typmod", typmodstr);
+ append_bool_object(typeParam, "typarray", typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ /*
+ * We don't use new_objtree_VA here because these names don't have a "fmt"
+ */
+ qualified = new_objtree();
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+ append_string_object(qualified, "schemaname", namespace);
+ append_string_object(qualified, "objname", pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *tmp;
+ ObjTree *ident;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+
+ if (alter_table)
+ tmp = new_objtree_VA("%{identity_type}s %{seq_definition: }s", 0);
+ else
+ tmp = new_objtree_VA("%{identity_type}s ( %{seq_definition: }s )", 0);
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(tmp, "present", false);
+ return tmp;
+ }
+
+ if (alter_table)
+ ident = new_objtree_VA("SET GENERATED %{option}s", 0);
+ else
+ ident = new_objtree_VA("GENERATED %{option}s AS IDENTITY", 0);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(ident, "option", "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(ident, "option", "BY DEFAULT");
+ else
+ append_bool_object(ident, "present", false);
+
+ append_object_object(tmp, "identity_type", ident);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(tmp, "seq_definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return tmp;
+}
+
+/*
+ * deparse_ColumnDef
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.)
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /* Composite types use a slightly simpler format string */
+ if (composite)
+ fmtstr = "%{name}I %{coltype}T %{collation}s";
+ else
+ fmtstr = "%{name}I %{coltype}T %{compression}s %{default}s %{not_null}s %{collation}s "
+ "%{generated_column}s %{identity_column}s";
+
+ column = new_objtree_VA(fmtstr, 3, "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree_VA("COMPRESSION %{compression_method}I", 0);
+ if (coldef->compression)
+ append_string_object(tmp, "compression_method", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "compression_method");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "compression", tmp);
+
+ tmp = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "name", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "collation", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "identity_column", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree_VA("GENERATED ALWAYS AS %{generation_expr}s STORED", 0);
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "generation_expr", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "generated_column", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparse_ColumnDef_Typed
+ * Subroutine for CREATE TABLE OF deparsing
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ *
+ * FIXME --- actually, what about default values?
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ column = new_objtree_VA("%{name}I WITH OPTIONS %{default}s %{not_null}s", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
+
+ tmp = new_objtree_VA("DEFAULT %{default}s", 0);
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "default", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "default", tmp);
+
+ /*
+ * Generated columns are not supported on typed tables, so we are done.
+ */
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * deparseTableElements
+ * Subroutine for CREATE TABLE deparsing
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparseTableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * obtainConstraints
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *tmp;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ tmp = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(tmp));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("ON COMMIT %{on_commit_value}s", 0);
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(tmp, "on_commit_value", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(tmp, "on_commit_value", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(tmp, "on_commit_value", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(tmp, "on_commit_value");
+ append_bool_object(tmp, "present", false);
+ break;
+ }
+
+ return tmp;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ if (elem->defnamespace != NULL)
+ optname = new_objtree_VA("%{schema}I.%{label}I", 1,
+ "schema", ObjTypeString, elem->defnamespace);
+ else
+ optname = new_objtree_VA("%{label}I", 0);
+
+ append_string_object(optname, "label", elem->defname);
+
+ if (is_reset)
+ set = new_objtree_VA("%{label}s", 0);
+ else
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ append_object_object(set, "label", optname);
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET (%{options:, }s)", 0);
+
+ append_string_object(tmp, "column", subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *tmp;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ if (is_reset)
+ tmp = new_objtree_VA("RESET (%{options:, }s)", 0);
+ else
+ tmp = new_objtree_VA("SET (%{options:, }s)", 0);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ append_array_object(tmp, "options", sets);
+
+ return tmp;
+}
+
+/*
+ * deparse_InhRelations
+ * Deparse the inherits relations
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+ char *sep;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ sep = "";
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, sep);
+ sep = ", ";
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* if it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+
+ /* all done */
+}
+
+/*
+ * deparse_IndexStmt
+ * deparse an IndexStmt
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ indexStmt =
+ new_objtree_VA("CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I "
+ "ON %{table}D USING %{index_am}s (%{definition}s) "
+ "%{with}s %{tablespace}s %{where_clause}s",
+ 6,
+ "unique", ObjTypeString, node->unique ? "UNIQUE" : "",
+ "concurrently", ObjTypeString,
+ node->concurrent ? "CONCURRENTLY" : "",
+ "if_not_exists", ObjTypeString, node->if_not_exists ? "IF NOT EXISTS" : "",
+ "name", ObjTypeString, RelationGetRelationName(idxrel),
+ "definition", ObjTypeString, definition,
+ "index_am", ObjTypeString, index_am);
+
+ tmp = new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel));
+ append_object_object(indexStmt, "table", tmp);
+
+ /* reloptions */
+ tmp = new_objtree_VA("WITH (%{opts}s)", 0);
+ if (reloptions)
+ append_string_object(tmp, "opts", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "with", tmp);
+
+ /* tablespace */
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}s", 0);
+ if (tablespace)
+ append_string_object(tmp, "tablespace", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "tablespace", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree_VA("WHERE %{where}s", 0);
+ if (whereClause)
+ append_string_object(tmp, "where", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "where_clause", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * deparse_CreateStmt
+ * Deparse a CreateStmt (CREATE TABLE)
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+ char *fmtstr;
+
+ /*
+ * Typed tables use a slightly different format string: we must not put
+ * table_elements with parents directly in the fmt string, because if
+ * there are no options the parens must not be emitted; and also, typed
+ * tables do not allow for inheritance.
+ */
+ if (node->ofTypename)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "OF %{of_type}T %{table_elements}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+ else if (node->partbound)
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "PARTITION OF %{parent_identity}D %{table_elements}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s %{partition_bound}s";
+ else
+ fmtstr = "CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D "
+ "(%{table_elements:, }s) %{inherits}s "
+ "%{partition_by}s %{access_method}s "
+ "%{with_clause}s %{on_commit}s %{tablespace}s";
+
+ createStmt =
+ new_objtree_VA(fmtstr, 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "identity", tmp);
+
+ append_string_object(createStmt, "if_not_exists",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ append_string_object(createStmt, "table_kind", "typed");
+
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "of_type", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_string_object(createStmt, "table_kind", "partition");
+
+ append_object_object(createStmt, "parent_identity",
+ elem->value.object);
+ }
+
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ tmp = new_objtree_VA("", 1,
+ "present", ObjTypeBool, false);
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "table_elements", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+ append_string_object(createStmt, "table_kind", "plain");
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparseTableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "table_elements", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree_VA("INHERITS (%{parents:, }D)", 0);
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "parents", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "parents");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "inherits", tmp);
+ }
+
+ tmp = new_objtree_VA("TABLESPACE %{tablespace}I", 0);
+ if (node->tablespacename)
+ append_string_object(tmp, "tablespace", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "tablespace");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "tablespace", tmp);
+
+ append_object_object(createStmt, "on_commit",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "partition_bound",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree_VA("PARTITION BY %{definition}s", 0);
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "definition", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "definition");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "partition_by", tmp);
+
+ /* USING clause */
+ tmp = new_objtree_VA("USING %{access_method}I", 0);
+ if (node->accessMethod)
+ append_string_object(tmp, "access_method", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "access_method");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "access_method", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree_VA("WITH (%{with:, }s)", 0);
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "with", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "with_clause", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+ ownerStmt = new_objtree_VA(fmt, 0);
+ append_string_object(ownerStmt, "newowner",
+ get_rolespec_name(node->newowner));
+
+ append_string_object(ownerStmt, "identity",
+ getObjectIdentity(&address, false));
+
+ return ownerStmt;
+}
+
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_SCHEMA:
+ {
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 0);
+ append_string_object(renameStmt, "identity", node->subname);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ append_string_object(renameStmt, "newname", node->newname);
+
+ return renameStmt;
+}
+
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ tmp = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ tmp = new_objtree_VA(fmt,
+ 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *tmp;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ tmp = new_objtree_VA("RESTART %{value}s",
+ 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(tmp);
+}
+
+/*
+ * deparse_CreateSchemaStmt
+ * deparse a CreateSchemaStmt
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ createSchema =
+ new_objtree_VA("CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s",
+ 2,
+ "name", ObjTypeString, node->schemaname,
+ "if_not_exists", ObjTypeString,
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+ auth = new_objtree_VA("AUTHORIZATION %{authorization_role}I", 0);
+ if (node->authrole)
+ append_string_object(auth, "authorization_role",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "authorization_role");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "authorization", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/* deparse_CreateSeqStmt
+ * deparse a CreateSeqStmt
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ createSeq =
+ new_objtree_VA("CREATE %{persistence}s SEQUENCE %{identity}D "
+ "%{definition: }s",
+ 1,
+ "persistence", ObjTypeString,
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "identity", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * deparse_AlterSeqStmt
+ * deparse an AlterSeqStmt
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq =
+ new_objtree_VA("ALTER SEQUENCE %{identity}D %{definition: }s", 0);
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "identity", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "definition", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
+ }
+
+ fmtstr = psprintf("ALTER %s %%{identity}D %%{subcmds:, }s", reltype);
+ alterTableStmt = new_objtree_VA(fmtstr, 0);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "identity", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX %index_name}I %{deferrable}s %{init_deferred}s",
+ 4, "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "constraint_type", ObjTypeString,
+ istmt->primary ? "PRIMARY KEY" : "UNIQUE",
+ "index_name", ObjTypeString, idxname);
+
+ append_string_object(tmp, "deferrable", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 1, "type", ObjTypeString, "drop default");
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT %{definition}s",
+ 1, "type", ObjTypeString, "set default");
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "definition",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+ append_string_object(tmp, "column", subcmd->name);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 1, "type", ObjTypeString, "drop not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 1, "type", ObjTypeString, "set not null");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION %{if_not_exists}s",
+ 2, "type", ObjTypeString, "set not null",
+ "if_not_exists", ObjTypeString,
+ subcmd->missing_ok ? "IF EXISTS": "");
+ append_string_object(tmp, "column", subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3, "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3, "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3, "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I %%{cascade}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "cascade", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3, "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* if no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 2, "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+ append_string_object(tmp, "deferrable", c->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "init_deferred", c->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I", 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE %%{datatype}T %%{collation}s %s",
+ istype ? "ATTRIBUTE" : "COLUMN",
+ istype ? "%{cascade}s" : "%{using}s");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* add the TYPE clause */
+ append_object_object(tmp, "datatype",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* add a COLLATE clause, if needed */
+ tmp2 = new_objtree_VA("COLLATE %{name}D", 0);
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "name", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "collation", tmp2);
+
+ /* if not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree_VA("USING %{expression}s", 0);
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "expression",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "using", tmp2);
+ }
+
+ /* if it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree_VA("CASCADE", 0);
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "cascade", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I",
+ 2, "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D",
+ 2, "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T",
+ 2, "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF",
+ 1, "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY %{ident}s", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "ident", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "ident", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "ident", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "ident", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D %{partition_bound}s",
+ 3, "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId),
+ "partition_bound", ObjTypeString,
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2, "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2, "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2, "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I %{definition}s",
+ 2, "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "definition", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY %{if_not_exists}s",
+ 3, "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name,
+ "if_not_exists", ObjTypeString,
+ subcmd->missing_ok ? "IF EXISTS" : "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "subcmds", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+char *
+deparse_drop_sequence(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SEQUENCE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_schema(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP SCHEMA IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_index(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP INDEX IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_drop_table(const char *objidentity, DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ ObjTree *stmt, *stmt2;
+ char *fmt;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+ fmt = psprintf("DROP TABLE IF EXISTS %%{objidentity}s %%{cascade}s");
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString,
+ objidentity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+ append_object_object(stmt, "cascade", stmt2);
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+
+ return command;
+}
+
+char *
+deparse_utility_command(CollectedCommand *cmd)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, 128);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..92564f1
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,763 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo out, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvBool, an error is raised. If it doesn't
+ * exist, tv_absent is returned; otherwise return the actual trivalue.
+ */
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo out)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(out, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
+ expand_jsonb_array(out, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(out, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar %= sizeof(dqsuffixes) - 1;
+ }
+ /* add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(out, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(out, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(out, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(out, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(out, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(out, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(out, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(out, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo out, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ size_t arrayseplen;
+ bool first = true;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if ((container->header & JB_FARRAY) == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ arrayseplen = strlen(arraysep);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo out = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(out);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, out);
+
+ return out->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index ddf219b..ce16206 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1699,6 +1699,40 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ /* open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..ee48470 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5189,11 +5189,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
- context);
+ address = ATExecAttachPartition(wqueue, rel,
+ (PartitionCmd *) cmd->def,
+ context);
else
- ATExecAttachPartitionIdx(wqueue, rel,
- ((PartitionCmd *) cmd->def)->name);
+ address = ATExecAttachPartitionIdx(wqueue, rel,
+ ((PartitionCmd *) cmd->def)->name);
break;
case AT_DetachPartition:
cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode,
@@ -5201,12 +5202,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
Assert(cmd != NULL);
/* ATPrepCmd ensures it must be a table */
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- ATExecDetachPartition(wqueue, tab, rel,
- ((PartitionCmd *) cmd->def)->name,
- ((PartitionCmd *) cmd->def)->concurrent);
+ address = ATExecDetachPartition(wqueue, tab, rel,
+ ((PartitionCmd *) cmd->def)->name,
+ ((PartitionCmd *) cmd->def)->concurrent);
break;
case AT_DetachPartitionFinalize:
- ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
+ address = ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
diff --git a/src/backend/utils/adt/format_type.c b/src/backend/utils/adt/format_type.c
index 060fd7e..6af2257 100644
--- a/src/backend/utils/adt/format_type.c
+++ b/src/backend/utils/adt/format_type.c
@@ -27,7 +27,6 @@
#include "utils/numeric.h"
#include "utils/syscache.h"
-static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
/*
@@ -367,7 +366,7 @@ format_type_with_typemod(Oid type_oid, int32 typemod)
/*
* Add typmod decoration to the basic type name
*/
-static char *
+char *
printTypmod(const char *typname, int32 typmod, Oid typmodout)
{
char *res;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 26cf4fa..35412f8 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -487,8 +487,6 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
@@ -502,7 +500,6 @@ static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1890,6 +1887,12 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2143,6 +2146,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -12017,7 +12029,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12797,7 +12809,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a77b293..f0db836 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11886,4 +11886,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 9da2300..2875106 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..bbfa872
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.h
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/ddl_deparse.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_table(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_sequence(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_schema(const char *objidentity, DropBehavior behavior);
+extern char *deparse_drop_index(const char *objidentity, DropBehavior behavior);
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 221c3e6..85a62a1 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid, int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
extern int32 type_maximum_size(Oid type_oid, int32 typemod);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..442d1ed 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -26,9 +26,11 @@ extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -41,6 +43,9 @@ extern List *select_rtable_names_for_explain(List *rtable,
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
--
2.7.2.windows.1
[application/octet-stream] v11-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (14.8K, ../../OS0PR01MB5716D3B79A607B547F2AB90D94819@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v11-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From e9da7a0ac55c6027687f8aadaff270df1341dea3 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Wed, 29 Jun 2022 17:04:32 +0800
Subject: [PATCH] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 18 ++++
src/backend/commands/event_trigger.c | 172 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 226 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index 2b33468..9fda2e9 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -3191,6 +3191,21 @@ deparse_AlterTableStmt(CollectedCommand *cmd)
return alterTableStmt;
}
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
/*
* Handle deparsing of simple commands.
*
@@ -3381,6 +3396,9 @@ deparse_utility_command(CollectedCommand *cmd)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 5d63aaf..e9cb887 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -585,7 +587,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -868,6 +871,163 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ /* Do nothing if no command was collected. */
+ if (!currentEventTriggerState->currentCommand)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1148,7 +1308,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1872,6 +2033,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1889,6 +2051,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 12d4b2e..ff75999 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -966,6 +966,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -977,6 +982,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6a5bcde..4d003de 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index f7f7165..7fb8cb2 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 13a470b..9b818b9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11901,4 +11901,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v11-0002-Support-DDL-replication.patch (125.6K, ../../OS0PR01MB5716D3B79A607B547F2AB90D94819@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v11-0002-Support-DDL-replication.patch)
download | inline diff:
From 409089c2ff0674a38e222e4917677a9d4dc96b17 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 21 Jun 2022 17:41:27 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/access/transam/rmgr.c | 1 +
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 224 +++++++++++++-
src/backend/commands/publicationcmds.c | 100 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 52 +++-
src/backend/replication/logical/reorderbuffer.c | 139 ++++++++-
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/adt/ri_triggers.c | 2 +
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
33 files changed, 1595 insertions(+), 217 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c
index 8ed6924..312f117 100644
--- a/src/backend/access/transam/rmgr.c
+++ b/src/backend/access/transam/rmgr.c
@@ -27,6 +27,7 @@
#include "fmgr.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/message.h"
#include "replication/origin.h"
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index c365de3..989c49a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index f46f864..5d63aaf 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1539,6 +1542,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1572,7 +1576,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1592,6 +1596,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2179,3 +2184,220 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged
+ * tables can also be created and altered on the subscriber side. This
+ * makes it possible to directly replay the SET LOGGED command and the
+ * incoming rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ {
+ command = deparse_drop_table(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_TableDropEnd,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "sequence", 8) == 0)
+ {
+ command = deparse_drop_sequence(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "schema", 6) == 0)
+ {
+ command = deparse_drop_schema(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+ else if (strncmp(obj->objecttype, "index", 6) == 0)
+ {
+ command = deparse_drop_index(obj->objidentity, stmt->behavior);
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, DCT_ObjectDrop,
+ command, strlen(command) + 1);
+ }
+
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..12d4b2e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,49 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1119,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ee48470..ddcce3f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5219,7 +5219,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..eaec031 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -640,8 +640,8 @@ logicalrep_read_truncate(StringInfo in,
*/
void
logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz,
- const char *message)
+ bool transactional, const char *prefix,
+ Size sz, const char *message)
{
uint8 flags = 0;
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..346e19e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4282,8 +4391,7 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
/* read prefix */
memcpy(&prefix_size, data, sizeof(Size));
data += sizeof(Size);
- change->data.msg.prefix = MemoryContextAlloc(rb->context,
- prefix_size);
+ change->data.msg.prefix = MemoryContextAlloc(rb->context, prefix_size);
memcpy(change->data.msg.prefix, data, prefix_size);
Assert(change->data.msg.prefix[prefix_size - 1] == '\0');
data += prefix_size;
@@ -4299,6 +4407,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 38e3b1c..2c667bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 2cbca4a..4cb09f9 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1974,7 +2148,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2032,6 +2207,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2145,6 +2321,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2330,6 +2507,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/adt/ri_triggers.c b/src/backend/utils/adt/ri_triggers.c
index 51b3fdc..7d60aac 100644
--- a/src/backend/utils/adt/ri_triggers.c
+++ b/src/backend/utils/adt/ri_triggers.c
@@ -33,6 +33,7 @@
#include "catalog/pg_operator.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
+#include "commands/event_trigger.h"
#include "executor/executor.h"
#include "executor/spi.h"
#include "lib/ilist.h"
@@ -40,6 +41,7 @@
#include "parser/parse_coerce.h"
#include "parser/parse_relation.h"
#include "storage/bufmgr.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/datum.h"
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index f502df9..59c30d0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c871cb7..ae7ca5e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3866,6 +3866,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3881,23 +3882,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3913,6 +3920,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3936,6 +3944,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4015,6 +4025,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 88d92a0..8b9c9a5 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6047,7 +6047,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6082,6 +6082,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6169,6 +6173,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6185,6 +6190,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6198,6 +6204,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6246,6 +6255,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6260,6 +6271,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6270,6 +6283,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f0db836..13a470b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11892,4 +11892,13 @@
{ oid => '4643', descr => 'json to string',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..de58035 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-08 02:26 Peter Smith <smithpb2250@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 449+ messages in thread
From: Peter Smith @ 2022-07-08 02:26 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
Here are some review comments for the patch v11-0001:
======
1. Commit message
This provides base for logical replication of DDL statements. Currently,
this provides support for:
SUGGESTION
This provides a base for logical replication of DDL statements. Currently,
the patch has support for:
======
2. src/backend/commands/ddl_deparse.c - <general>
I noticed that some of these function header comments have periods and
so do not. Please add a period to every one of them for consistency.
~~~
3. src/backend/commands/ddl_deparse.c - <general>
There are quite a few functions in this file with no function comment.
Probably every function should have a comment.
List includes at least all these:
- deparse_ColumnIdentity
- RelationGetPartitionBound
- deparse_AlterOwnerStmt
- deparse_RenameStmt
- deparse_Seq_Cache
- deparse_Seq_Cycle
- deparse_Seq_IncrementBy
- deparse_Seq_Minvalue
- deparse_Seq_Maxvalue
- deparse_Seq_Startwith
- deparse_Seq_Restart
- deparse_Seq_OwnedBy
- deparse_AlterTableStmt
- deparse_CreateTableAsStmt
- deparse_drop_sequence
- deparse_drop_schema
- deparse_drop_index
- deparse_drop_table
~~~
4. src/backend/commands/ddl_deparse.c - <general>
Lots of places are making calls to the new_objtree_VA function but
some of them are a bit messy. I think the wrapping of the args to that
function needs to be revisited and made consistent indentation
everywhere to make them all easier to read. IMO it is easier when the
number of arg-groups is clear and each arg-group is on a new line.
Like this example:
column = new_objtree_VA("%{name}I WITH OPTIONS %{default}s %{not_null}s",
2,
"type", ObjTypeString, "column",
"name", ObjTypeString,
coldef->colname);
~~~
5. src/backend/commands/ddl_deparse.c - <general>
Lots of the function comments are giving the function name again. It
does not seem necessary.
e.g (there many more are like this)
BEFORE
/* deparse_CreateSeqStmt
* deparse a CreateSeqStmt
*
* Given a sequence OID and the parsetree that create it, return an ObjTree
* representing the creation command.
*/
SUGGEST
/*
* Deparse a CreateSeqStmt
*
* Given a sequence OID and the parsetree that create it, return an ObjTree
* representing the creation command.
*/
~~~
6. src/backend/commands/ddl_deparse.c - typedefs
6a.
+} ObjType;
Shouldn't this be added to typedefs.list?
~
6b.
+typedef struct ObjTree
Ditto.
~
6c.
+typedef struct ObjElem
Ditto
~~~
7. src/backend/commands/ddl_deparse.c - format_type_detailed
+ }
+ *nspid = InvalidOid;
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+ return;
+ }
+
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
The code can be simplified a bit by using if/else because the part:
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
is common code.
~~~
8. src/backend/commands/ddl_deparse.c - append_bool_object
Why this function has no assert like the others do?
+ Assert(name);
~~~
9. src/backend/commands/ddl_deparse.c - append_array_object
Why this function has no assert like the others do?
+ Assert(name);
~~~
10. src/backend/commands/ddl_deparse.c - new_objtree_for_type
+ if (!OidIsValid(typnspid))
+ typnsp = pstrdup("");
+ else
+ typnsp = get_namespace_name_or_temp(typnspid);
Reversing this if/else will give slight simpler code
~~~
11. src/backend/commands/ddl_deparse.c - deparse_ColumnIdentity
+ ObjTree *tmp;
"tmp" doesn’t seem a very good variable name since that is also what
the function is returning.
~~~
12.
+ /* definition elemets */
Uppercase comment.
~~~
13.
+ /* we purposefully do not emit OWNED BY here */
Uppercase comment.
~~~
14. src/backend/commands/ddl_deparse.c - deparse_ColumnDef
+ /*
+ * Inherited columns without local definitions must not be emitted. XXX --
+ * maybe it is useful to have them with "present = false" or some such?
+ */
I think the XXX should be on a newline otherwise the note just gets
lost in the comment.
~~~
15.
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
Perhaps simple code like this is more neatly written as:
append_string_object(column, "not_null", saw_notnull ? "NOT NULL" : "");
~~~
16. src/backend/commands/ddl_deparse.c - deparse_ColumnDef_typed
+ if (saw_notnull)
+ append_string_object(column, "not_null", "NOT NULL");
+ else
+ append_string_object(column, "not_null", "");
Ditto previous comment #15.
~~~
17. src/backend/commands/ddl_deparse.c - deparseTableElements
Should the function name be "deparse_TableElements" to match the
pattern of all the others?
~~~
18. src/backend/commands/ddl_deparse.c - deparse_OnCommitClause
+ ObjTree *tmp;
I don’t think "tmp" is a good name here because the function is returning it.
~~~
19. src/backend/commands/ddl_deparse.c - deparse_DefElem
+ set = new_objtree_VA("%{label}s = %{value}L", 1,
+ "value", ObjTypeString,
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
The double ternary operators here are a bit hard to read. Maybe add
some extra parens just to improve the readability?
~~~
20. src/backend/commands/ddl_deparse.c - deparse_ColumnSetOptions
+ ObjTree *tmp;
I don’t think "tmp" is a very good name here because the function is
returning this.
~~~
21. src/backend/commands/ddl_deparse.c - deparse_RelSetOptions
+ ObjTree *tmp;
I don’t think "tmp" is a very good name here because the function is
returning this.
~~~
22 src/backend/commands/ddl_deparse.c - pg_get_indexdef_detailed
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
Spurious whitespace line.
~~~
23.
+ /* output index AM */
Uppercase comment
~~~
24.
+ sep = "";
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, sep);
+ sep = ", ";
This assignment of "sep" seemed a bit strange. Can't it be more
easilly written like:
appendStringInfoString(&definitionBuf, keyno == 0 ? "" : ", ");
~~~
25.
+ /* expressional index */
Uppercase comment.
~~~
26.
+ /* if it supports sort ordering, report DESC and NULLS opts */
Uppercase comment
~~~
27.
+ /* output tablespace */
Uppercase comment
~~~
28.
+ /* report index predicate, if any */
Uppercase comment
~~~
29.
+
+ /* all done */
Kind of redundant/meaningless. Consider removing this comment.
~~~
30. src/backend/commands/ddl_deparse.c - deparse_RenameStmt
+ switch (node->renameType)
+ {
+ case OBJECT_SCHEMA:
+ {
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 0);
+ append_string_object(renameStmt, "identity", node->subname);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
The switch with single case seems a bit overkill here. Wouldn’t just
"if" be more appropriate?
~~~
31. src/backend/commands/ddl_deparse.c - deparse_CreateSeqStmt
This function looked very similar to the other function
deparse_ColumnIdentity. Is it worth trying to combine these or have
one of them just delegate to the other to reduce the cut/paste code?
~~~
32. src/backend/commands/ddl_deparse.c - deparse_AlterTableStmt
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ reltype = NULL;;
Does the assignment after the elog achieve anything?
~~~
33.
+ /* if no constraint was altered, silently skip it */
Uppercase comment
~~~
34.
+ /* add the TYPE clause */
Uppercase comment
~~~
35.
+ /* add a COLLATE clause, if needed */
Uppercase comment
~~~
36.
+ /* if not a composite type, add the USING clause */
Uppercase comment
~~~
37.
+ /* if it's a composite type, add the CASCADE clause */
Uppercase comment
~~~
38. src/backend/commands/ddl_deparse.c - deparse_drop_sequence
38a.
+ command = JsonbToCString(&str, &jsonb->root, 128);
Is there some more appropriate constant to use here instead of the
hardwired 128?
~~
38b. deparse_drop_schema
ditto
~~
38c. deparse_drop_index
ditto
~~
38d. deparse_drop_table
ditto
~~~
39. src/backend/commands/ddl_deparse.c - ddl_deparse_to_json
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+ else
+ PG_RETURN_NULL();
The 'else’ keyword is not needed. Just do PG_RETURN_NULL();
======
40. src/backend/commands/ddl_json.c - <general>
Many (but not all) of these comments (particularly the function header
comments) seem to have double blank spaces in them after periods. I
don’t think it is normal. Please remove the extra spaces
~~~
41. src/backend/commands/ddl_json.c - <general>
All the function header comment sentences should start with uppercase. E.g. in
many places:
"expand" -> "Expand"
~~~
42. src/backend/commands/ddl_json.c -
42.a
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
Add this to typedefs.list?
~~
42.b
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
Add this to typedefs.list?
~~~
43. src/backend/commands/ddl_json.c - find_string_in_jsonbcontainer
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised if missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
"an error is raised if missing_ok" --> I think this should say an
error is raised *unless* missing_ok
~~~
44.
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
For some reason, it seems more intuitive IMO to handle the error case
first here. YMMV.
if (value == NULL)
{
if (!missing_ok)
ereport(...)
return NULL;
}
~~~
45. src/backend/commands/ddl_json.c - expand_fmt_recursive
+ /*
+ * found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
Uppercase comment
~~~
46.
+ /* Validate that we got an array if the format string specified one. */
+
+ /* And finally print out the data */
+ if (is_array)
Something seems strange for these comments to be separated like this.
~~~
47 src/backend/commands/ddl_json.c - expand_jsonval_identifier
+ str = pnstrdup(jsonval->val.string.val,
+ jsonval->val.string.len);
Unnecessary wrapping
~~~
48. src/backend/commands/ddl_json.c - expand_jsonval_dottedname
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and must contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
Does it make sense to say "must" contain elements that are optional elements?
~~~
49. src/backend/commands/ddl_json.c - expand_jsonval_strlit
+ /* easy case: if there are no ' and no \, just use a single quote */
Uppercase comment
~~~
50.
+ dqnextchar %= sizeof(dqsuffixes) - 1;
This statement looks quite confusing. May add some parens for better reability?
~~~
51.
+ /* add trailing $ */
Uppercase comment.
~~~
52. src/backend/commands/ddl_json.c - expand_one_jsonb_element
+static bool
+expand_one_jsonb_element(StringInfo out, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
In the other function the StringInfo buffer was called "buf" instead
of "out". Either one is OK but I think you should use consistent
naming for all the functions like this. Check all the function – not
just this one.
~~~
53. src/backend/commands/ddl_json.c - expand_jsonb_array
+ if ((container->header & JB_FARRAY) == 0)
I think there is a macro you should use designed exactly for this –
see JsonContainerIsArray(jc)
~~~
54.
+ switch (type)
+ {
+ case WJB_ELEM:
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
+ break;
+ }
Why have a switch with just a single case?
~~~
55.
+ if (!first)
+ appendStringInfoString(out, arraysep);
+
+ if (expand_one_jsonb_element(out, param, &v, specifier, NULL))
+ first = false;
+ else
+ {
+ if (!first)
+ {
+ /* remove the array separator */
+ out->len -= arrayseplen;
+ out->data[out->len] = '\0';
+ }
+ }
It looks a bit strange to first write the separator to the buffer then
find that you shouldn't have written it so have to undo it. Is this
the best way that this can be handled? Maybe there is no choice.
~~~
56.
+ /* remove the array separator */
Uppercase comment
~~~~
57. src/backend/commands/ddl_json.c - ddl_deparse_json_to_string
+char *
+ddl_deparse_json_to_string(char *json_str)
Missing function comment
~~~
58. src/backend/commands/ddl_json.c - ddl_deparse_expand_command
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
This function name seems a bit strange - see also comment #62
======
59. src/backend/commands/sequence.c - get_sequence_values
+ /* open and AccessShareLock sequence */
Uppercase comment
~~~
60.
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ /* open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
I thought maybe better if the palloc could be done later, paired with
the memcpy, since it is not needed before then.
======
61. src/backend/utils/adt/ruleutils.c - pg_get_partkeydef_simple
+char *
+pg_get_partkeydef_simple(Oid relid)
Missing function comment
======
62. src/include/catalog/pg_proc.dat
+{ oid => '4642', descr => 'ddl deparse',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'json to string',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
My 1st impressions was the name "ddl_deparse_expand_command" does not
see to reflext the description very well...
Maybe calling it something like "ddl_json_to_command" is more accurate?
======
63. src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid,
int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
Notice that every of the format_type function name looks like
format_type_XXX. Not that you have change the printTypmod to be extern
then I woinder should the name also be changed (e.g.
format_type_printmod) to have the consistent function naming.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-07-12 09:03 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Peter Smith <smithpb2250@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-07-12 09:03 UTC (permalink / raw)
To: Peter Smith <smithpb2250@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Friday, July 8, 2022 10:26 AM Peter Smith <smithpb2250@gmail.com> wrote:
> Here are some review comments for the patch v11-0001:
Thanks for the comments.
> 4. src/backend/commands/ddl_deparse.c - <general>
>
> Lots of places are making calls to the new_objtree_VA function but some of them are a bit messy. I think the > wrapping of the args to that function needs to be revisited and made consistent indentation everywhere to make them > all easier to read. IMO it is easier when the number of arg-groups is clear and each arg-group is on a new line.
> Like this example:
>
> column = new_objtree_VA("%{name}I WITH OPTIONS %{default}s %{not_null}s",
> 2,
> "type", ObjTypeString, "column",
> "name", ObjTypeString,
> coldef->colname);
I think both your suggestion and the following style are fine to me.
new_objtree_VA(fmt, num,
xxobj1,
xxobj2 );
So, I only changed the other style function calls.
> 30. src/backend/commands/ddl_deparse.c - deparse_RenameStmt
>
> + switch (node->renameType)
> + {
> + case OBJECT_SCHEMA:
> + {
> + renameStmt =
> + new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
> + 0);
> + append_string_object(renameStmt, "identity", node->subname); } break;
> + default:
> + elog(ERROR, "unsupported object type %d", node->renameType); }
> The switch with single case seems a bit overkill here. Wouldn’t just "if" be
> more appropriate?
I think it was intended for other RENAME deparsing which haven't been added but
will be added to the patch later. So, I didn't change this.
> 31. src/backend/commands/ddl_deparse.c - deparse_CreateSeqStmt
>
> This function looked very similar to the other function deparse_ColumnIdentity.
> Is it worth trying to combine these or have one of them just delegate to the
> other to reduce the cut/paste code?
Since they are used to deparse different commands, one for (column identity)
another for (create sequence), so I think the current style is fine.
> 40. src/backend/commands/ddl_json.c - <general>
>
> Many (but not all) of these comments (particularly the function header
> comments) seem to have double blank spaces in them after periods. I don’t
> think it is normal. Please remove the > extra spaces
I think this style is fine as I can see them in many other existing comments.
> 44.
> + if (value == NULL)
> + {
> + if (missing_ok)
> + return NULL;
> + ereport(ERROR,
> + (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
> + errmsg("missing element \"%s\" in json object", keyname))); }
>
> For some reason, it seems more intuitive IMO to handle the error case first here. YMMV.
Not sure about this.
> 62. src/include/catalog/pg_proc.dat
>
> +{ oid => '4642', descr => 'ddl deparse',
> + proname => 'ddl_deparse_to_json', prorettype => 'text',
> + proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' }, {
> +oid => '4643', descr => 'json to string',
> + proname => 'ddl_deparse_expand_command', prorettype => 'text',
> + proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
>
> My 1st impressions was the name "ddl_deparse_expand_command" does not see to reflext the description very well...
>
> Maybe calling it something like "ddl_json_to_command" is more accurate ?
I think "expand command" is close to the usage of this function so didn't change the
name for now. But I adjust the description here.
> 63. src/include/utils/builtins.h
>
> @@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid,
> int32 typemod, bits16 flags);
> extern char *format_type_be(Oid type_oid); extern char *format_type_be_qualified(Oid type_oid); extern char > *format_type_with_typemod(Oid type_oid, int32 typemod);
> +extern char *printTypmod(const char *typname, int32 typmod, Oid
> +typmodout);
>
> Notice that every of the format_type function name looks like format_type_XXX.
> Not that you have change the printTypmod to be extern then I woinder should the
> name also be changed (e.g. format_type_printmod) to have the consistent
> function naming.
I am not sure about this as the current name looks fine to me.
I agreed with the other comments and addressed them on the new version patch.
Attach the V12 patch set which include the following changes:
* Address comments from peter[1]
* Refactor the deparser and provide an option to control whether output the "not present" syntax part.
And for DDL replication, we don't WAL log the "not present" syntax string for now.
* Address most comments from Vignesh[2] except the one about pg_dump
vs the event trigger for ddl replication which need some more research.
[1] https://www.postgresql.org/message-id/CAHut%2BPs9QyGw4KRFP50vRnB0tJKbB_TS1E7rZ_-%2Bpc2Nvwv_zw%40mail...
[2] https://www.postgresql.org/message-id/CALDaNm2nFPMxUo%3D0zRUUA-v3_eRwRY%2Bii5nnG_PU%2B6jT7ta9dA%40ma...
Best regards,
Hou zj
Attachments:
[application/octet-stream] v12-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB571642FDCB049FC00EBDA5EC94869@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v12-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From b25920a8213e38de9eb8bb096bdc29d64f3e9482 Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
1.8.3.1
[application/octet-stream] v12-0001-Functions-to-deparse-DDL-commands.patch (134.2K, ../../OS0PR01MB571642FDCB049FC00EBDA5EC94869@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v12-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 19bc4aec727d0c630d3be6895e37caa210851a30 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides a base for logical replication of DDL statements. Currently,
the patch has support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/ALTER/DROP SCHEMA
CREATE/DROP INDEX
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 3595 +++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 760 ++++++++
src/backend/commands/sequence.c | 33 +
src/backend/commands/tablecmds.c | 17 +-
src/backend/utils/adt/format_type.c | 3 +-
src/backend/utils/adt/ruleutils.c | 23 +-
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 23 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/builtins.h | 1 +
src/include/utils/ruleutils.h | 5 +
src/tools/pgindent/typedefs.list | 4 +
14 files changed, 4459 insertions(+), 15 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..22b023d
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,3595 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+/* Estimated length of the generated jsonb string */
+#define JSONB_ESTIMATED_LEN 128
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+ StringInfo fmtinfo;
+ bool present;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+bool verbose = true;
+
+static ObjElem *new_object(ObjType type, char *name);
+static ObjElem *new_object_object(ObjTree *value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_extended, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
+
+ /*
+ * Special-case crock for types with strange typmod rules where we put
+ * typmod at the middle of name(e.g. TIME(6) with time zone ). We cannot
+ * schema-qualify nor add quotes to the type name in these cases.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmod. */
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
+ break;
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+ }
+ else
+ {
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+ }
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(char *fmt)
+{
+ ObjTree *params;
+
+ params = palloc0(sizeof(ObjTree));
+ params->present = true;
+ slist_init(¶ms->params);
+
+ if (fmt)
+ {
+ params->fmtinfo = makeStringInfo();
+ appendStringInfoString(params->fmtinfo, fmt);
+ }
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree(fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+ elem = new_object(type, NULL);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ break;
+ case ObjTypeBool:
+ elem->value.boolean = va_arg(args, int);
+ break;
+ case ObjTypeString:
+ elem->value.string = va_arg(args, char *);
+ break;
+ case ObjTypeArray:
+ elem->value.array = va_arg(args, List *);
+ break;
+ case ObjTypeInteger:
+ elem->value.integer = va_arg(args, int64);
+ break;
+ case ObjTypeFloat:
+ elem->value.flt = va_arg(args, double);
+ break;
+ case ObjTypeObject:
+ elem->value.object = va_arg(args, ObjTree *);
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/*
+ * Append the input format string to the ObjTree.
+ */
+static void
+append_format_string(ObjTree *tree, char *sub_fmt)
+{
+ int len;
+ char *fmt;
+
+ if (!tree->fmtinfo)
+ return;
+
+ fmt = tree->fmtinfo->data;
+ len = tree->fmtinfo->len;
+
+ /* Add a separator if necessary */
+ if (len > 0 && fmt[len - 1] != ' ')
+ appendStringInfoSpaces(tree->fmtinfo, 1);
+
+ appendStringInfoString(tree->fmtinfo, sub_fmt);
+}
+
+/*
+ * Return the object name which is extracted from the input "*%{name[:.]}*"
+ * style string. And append the input format string to the ObjTree.
+ */
+static char *
+append_object_to_format_string(ObjTree *tree, char *sub_fmt)
+{
+ StringInfoData object_name;
+ const char *end_ptr;
+ const char *cp;
+ bool start_copy = false;
+
+ if (sub_fmt == NULL || tree->fmtinfo == NULL)
+ return sub_fmt;
+
+ initStringInfo(&object_name);
+ end_ptr = sub_fmt + strlen(sub_fmt);
+
+ for (cp = sub_fmt; cp < end_ptr; cp++)
+ {
+ if (*cp == '{')
+ {
+ start_copy = true;
+ continue;
+ }
+
+ if (!start_copy)
+ continue;
+
+ if (*cp == ':' || *cp == '}')
+ break;
+
+ appendStringInfoCharMacro(&object_name, *cp);
+ }
+
+ if (object_name.len == 0)
+ elog(ERROR, "object name not found");
+
+ append_format_string(tree, sub_fmt);
+
+ return object_name.data;
+}
+
+/* Allocate a new parameter */
+static ObjElem *
+new_object(ObjType type, char *name)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = name;
+ param->objtype = type;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *sub_fmt)
+{
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ append_premade_object(tree, new_object(ObjTypeNull, object_name));
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *sub_fmt, bool value)
+{
+ ObjElem *param;
+ char *object_name = sub_fmt;
+ bool is_present_flag = false;
+
+ Assert(sub_fmt);
+
+ if (strcmp(sub_fmt, "present") == 0)
+ {
+ is_present_flag = true;
+ tree->present = value;
+ }
+
+ if (!verbose && !tree->present)
+ return;
+
+ if (!is_present_flag)
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeBool, object_name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
+
+/* Append a string parameter to a tree */
+static void
+append_string_object(ObjTree *tree, char *sub_fmt, char *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && (value == NULL || value[0] == '\0'))
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeString, object_name);
+ param->value.string = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeObject, NULL);
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *sub_fmt, ObjTree *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && !value->present)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeObject, object_name);
+ param->value.object = value;
+ append_premade_object(tree, param);
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *sub_fmt, List *array)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && list_length(array) == 0)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeArray, object_name);
+ param->value.array = array;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Process the pre-built format string from the ObjTree into the output parse
+ * state.
+ */
+static void
+objtree_fmt_to_jsonb_element(JsonbParseState *state, ObjTree *tree)
+{
+ JsonbValue key;
+ JsonbValue val;
+
+ if (!tree->fmtinfo)
+ return;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.val = "fmt";
+ key.val.string.len = strlen(key.val.string.val);
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the pre-built format string */
+ val.type = jbvString;
+ val.val.string.len = tree->fmtinfo->len;
+ val.val.string.val = tree->fmtinfo->data;
+ pushJsonbValue(&state, WJB_VALUE, &val);
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb.
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ objtree_fmt_to_jsonb_element(state, tree);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (OidIsValid(typnspid))
+ typnsp = get_namespace_name_or_temp(typnspid);
+ else
+ typnsp = pstrdup("");
+
+ typeParam = new_objtree_VA(NULL, 4,
+ "schemaname", ObjTypeString, typnsp,
+ "typename", ObjTypeString, typename,
+ "typmod", ObjTypeString, typmodstr,
+ "typarray", ObjTypeBool, typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements.
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+
+ qualified = new_objtree_VA(NULL, 2,
+ "schemaname", ObjTypeString, namespace,
+ "objname", ObjTypeString, pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId.
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value.
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * Deparse the definition of column identity.
+ */
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *column;
+ ObjTree *identobj;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+ char *identfmt;
+ char *objfmt;
+
+ column = new_objtree("");
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(column, "present", false);
+ return column;
+ }
+
+ if (alter_table)
+ {
+ identfmt = "SET GENERATED ";
+ objfmt = "%{option}s";
+ }
+ else
+ {
+ identfmt = "GENERATED ";
+ objfmt = "%{option}s AS IDENTITY";
+ }
+
+ identobj = new_objtree(identfmt);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(identobj, objfmt, "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(identobj, objfmt, "BY DEFAULT");
+ else
+ append_bool_object(identobj, "present", false);
+
+ append_object_object(column, "%{identity_type}s", identobj);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* Definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* We purposefully do not emit OWNED BY here */
+
+ if (alter_table)
+ append_array_object(column, "%{seq_definition: }s", elems);
+ else
+ append_array_object(column, "( %{seq_definition: }s )", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.).
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted.
+ *
+ * XXX maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I %{coltype}T %{compression}s %{default}s %{not_null}s
+ * %{collation}s
+ */
+ column = new_objtree_VA("%{name}I %{coltype}T", 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree("COMPRESSION");
+ if (coldef->compression)
+ append_string_object(tmp, "%{compression_method}I", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "%{compression_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "%{compression}s", tmp);
+
+ tmp = new_objtree("COLLATE");
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{collation}s", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "%{identity_column}s", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree("GENERATED ALWAYS AS");
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "%{generation_expr}s STORED", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "%{generated_column}s", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE OF deparsing.
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I WITH OPTIONS %{default}s %{not_null}s.
+ */
+ column = new_objtree_VA("%{name}I WITH OPTIONS", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /*
+ * Generated columns are not supported on typed tables, so we are done.
+ */
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparse_TableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing.
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *constr;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ constr = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(constr));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * Deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *oncommit;
+
+ oncommit = new_objtree("ON COMMIT");
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(oncommit, "%{on_commit_value}s", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(oncommit, "%{on_commit_value}s");
+ append_bool_object(oncommit, "present", false);
+ break;
+ }
+
+ return oncommit;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ set = new_objtree("");
+ optname = new_objtree("");
+
+ if (elem->defnamespace != NULL)
+ append_string_object(optname, "%{schema}I.", elem->defnamespace);
+
+ append_string_object(optname, "%{label}I", elem->defname);
+
+ append_object_object(set, "%{label}s", optname);
+
+ if (!is_reset)
+ append_string_object(set, " = %{value}L",
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *colset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER COLUMN %{column}I RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "ALTER COLUMN %{column}I RESET ";
+ else
+ fmt = "ALTER COLUMN %{column}I SET ";
+
+ colset = new_objtree_VA(fmt, 1, "column", ObjTypeString, subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(colset, "(%{options:, }s)", sets);
+
+ return colset;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *relset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "RESET ";
+ else
+ fmt = "SET ";
+
+ relset = new_objtree(fmt);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(relset, "(%{options:, }s)", sets);
+
+ return relset;
+}
+
+/*
+ * Deparse the inherits relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+/*
+ * Obtain the deparsed partition bound expression for the given table.
+ */
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* Output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, keyno == 0 ? "" : ", ");
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* Expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* If it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* Output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* Output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* Report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+}
+
+/*
+ * Deparse an IndexStmt.
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I ON
+ * %{table}D USING %{index_am}s (%{definition}s) %{with}s %{tablespace}s
+ * %{where_clause}s
+ */
+ indexStmt = new_objtree("CREATE");
+
+ append_string_object(indexStmt, "%{unique}s",
+ node->unique ? "UNIQUE" : "");
+
+ append_format_string(indexStmt, "INDEX");
+
+ append_string_object(indexStmt, "%{concurrently}s",
+ node->concurrent ? "CONCURRENTLY" : "");
+
+ append_string_object(indexStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(indexStmt, "%{name}I",
+ RelationGetRelationName(idxrel));
+
+ append_object_object(indexStmt, "ON %{table}D",
+ new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel)));
+
+ append_string_object(indexStmt, "USING %{index_am}s", index_am);
+
+ append_string_object(indexStmt, "(%{definition}s)", definition);
+
+ /* reloptions */
+ tmp = new_objtree("WITH");
+ if (reloptions)
+ append_string_object(tmp, "(%{opts}s)", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{with}s", tmp);
+
+ /* tablespace */
+ tmp = new_objtree("TABLESPACE");
+ if (tablespace)
+ append_string_object(tmp, "%{tablespace}s", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{tablespace}s", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree("WHERE");
+ if (whereClause)
+ append_string_object(tmp, "%{where}s", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{where_clause}s", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * Deparse a CreateStmt (CREATE TABLE).
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D [OF
+ * %{of_type}T | PARTITION OF %{parent_identity}D] %{table_elements}s
+ * %{inherits}s %{partition_by}s %{access_method}s %{with_clause}s
+ * %{on_commit}s %{tablespace}s
+ */
+ createStmt = new_objtree("CREATE");
+
+ append_string_object(createStmt, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ append_format_string(createStmt, "TABLE");
+
+ append_string_object(createStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "%{identity}D", tmp);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ /*
+ * Typed tables and partitions use a slightly different format string: we
+ * must not put table_elements with parents directly in the fmt string,
+ * because if there are no options the parens must not be emitted; and
+ * also, typed tables do not allow for inheritance.
+ */
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "OF %{of_type}T", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_object_object(createStmt, "%{parent_identity}D",
+ elem->value.object);
+ }
+
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ {
+ tmp = new_objtree("");
+ append_bool_object(tmp, "present", false);
+ }
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "(%{table_elements:, }s)", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "(%{table_elements:, }s)", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree("INHERITS");
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "(%{parents:, }D)", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "(%{parents:, }D)");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{inherits}s", tmp);
+ }
+
+ tmp = new_objtree("TABLESPACE");
+ if (node->tablespacename)
+ append_string_object(tmp, "%{tablespace}I", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "%{tablespace}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{tablespace}s", tmp);
+
+ append_object_object(createStmt, "%{on_commit}s",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "%{partition_bound}s",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree("PARTITION BY");
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "%{definition}s", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "%{definition}s");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{partition_by}s", tmp);
+
+ /* USING clause */
+ tmp = new_objtree("USING");
+ if (node->accessMethod)
+ append_string_object(tmp, "%{access_method}I", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "%{access_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{access_method}s", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree("WITH");
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "(%{with:, }s)", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "%{with_clause}s", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+/*
+ * Deparse a AlterOwnerStmt (ALTER ... OWNER TO ...).
+ */
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+
+ ownerStmt = new_objtree_VA(fmt, 2,
+ "identity", ObjTypeString,
+ getObjectIdentity(&address, false),
+ "newowner", ObjTypeString,
+ get_rolespec_name(node->newowner));
+
+ return ownerStmt;
+}
+
+/*
+ * Deparse a RenameStmt.
+ */
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_SCHEMA:
+ {
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO",
+ 1,
+ "identity", ObjTypeString, node->subname);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ append_string_object(renameStmt, "%{newname}I", node->newname);
+
+ return renameStmt;
+}
+
+/*
+ * Deparse the sequence CACHE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cache;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ cache = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(cache);
+}
+
+/*
+ * Deparse the sequence CYCLE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cycle;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ cycle = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(cycle);
+}
+
+/*
+ * Deparse the sequence INCREMENT BY option.
+ */
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *incrementalby;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ incrementalby = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(incrementalby);
+}
+
+/*
+ * Deparse the sequence MINVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *minvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ minvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(minvalue);
+}
+
+/*
+ * Deparse the sequence MAXVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *maxvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ maxvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(maxvalue);
+}
+
+/*
+ * Deparse the sequence START WITH option.
+ */
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *startwith;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ startwith = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(startwith);
+}
+
+/*
+ * Deparse the sequence RESTART option.
+ */
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *restart;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ restart = new_objtree_VA("RESTART %{value}s", 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(restart);
+}
+
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s
+ */
+ createSchema = new_objtree("CREATE SCHEMA");
+
+ append_string_object(createSchema, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(createSchema, "%{name}I", node->schemaname);
+
+ auth = new_objtree("AUTHORIZATION");
+ if (node->authrole)
+ append_string_object(auth, "%{authorization_role}I",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "%{authorization_role}I ");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "%{authorization}s", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/*
+ * Deparse a CreateSeqStmt.
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s SEQUENCE %{identity}D
+ */
+ createSeq = new_objtree("CREATE");
+
+ append_string_object(createSeq, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "SEQUENCE %{identity}D", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * Deparse an AlterSeqStmt.
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq = new_objtree("ALTER SEQUENCE");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "%{identity}D", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+/*
+ * Deparse all the collected subcommands and return an ObjTree representing the
+ * alter command.
+ */
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER reltype %{identity}D %{subcmds:, }s
+ */
+ fmtstr = psprintf("ALTER %s", reltype);
+ alterTableStmt = new_objtree(fmtstr);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "%{identity}D", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+
+ /*
+ * Verbose syntax
+ *
+ * ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX
+ * %index_name}I %{deferrable}s %{init_deferred}s
+ */
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I",
+ 2,
+ "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+
+ append_string_object(tmp, "%{constraint_type}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "USING INDEX %{index_name}I",
+ RelationGetRelationName(idx));
+ append_string_object(tmp, "%{deferrable}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "%{init_deferred}s", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 2,
+ "type", ObjTypeString, "drop default",
+ "column", ObjTypeString, subcmd->name);
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT",
+ 2,
+ "type", ObjTypeString, "set default",
+ "column", ObjTypeString, subcmd->name);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "%{definition}s",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 2,
+ "type", ObjTypeString, "drop not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 2,
+ "type", ObjTypeString, "set not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION",
+ 2,
+ "type", ObjTypeString, "drop expression",
+ "column", ObjTypeString, subcmd->name);
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS": "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3,
+ "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3,
+ "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* If no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 4,
+ "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "deferrable", ObjTypeString,
+ c->deferrable ? "DEFERRABLE" : "NOT DEFERRABLE",
+ "init_deferred", ObjTypeString,
+ c->initdeferred ? "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s ATTRIBUTE %{cascade}s
+ *
+ * Normal types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s COLUMN %{using}s
+ */
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE",
+ istype ? "ATTRIBUTE" : "COLUMN");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* Add the TYPE clause */
+ append_object_object(tmp, "%{datatype}T",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* Add a COLLATE clause, if needed */
+ tmp2 = new_objtree("COLLATE");
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{collation}s", tmp2);
+
+ /* If not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree("USING");
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "%{expression}s",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{using}s", tmp2);
+ }
+
+ /* If it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree("CASCADE");
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I", 2,
+ "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T", 2,
+ "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF", 1,
+ "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "%{ident}s", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "%{ident}s", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "%{ident}s", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "%{ident}s", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D %{partition_bound}s",
+ 3,
+ "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId),
+ "partition_bound", ObjTypeString,
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2,
+ "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2,
+ "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2,
+ "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I",
+ 2,
+ "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "%{definition}s", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY",
+ 2,
+ "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name);
+
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS" : "");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "%{subcmds:, }s", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+/*
+ * Handle deparsing of DROP commands.
+ */
+char *
+deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ char *identity = (char *) objidentity;
+ char *fmt;
+ ObjTree *stmt, *stmt2;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+
+ fmt = psprintf("DROP %s IF EXISTS %%{objidentity}s", objecttype);
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString, identity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+
+ append_object_object(stmt, "%{cascade}s", stmt2);
+
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+
+ return command;
+}
+
+/*
+ * Workhorse to deparse a CollectedCommand.
+ */
+char *
+deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ verbose = verbose_mode;
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd, true);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..d64719b
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,760 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo buf, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvBool, an error is raised. If it doesn't
+ * exist, tv_absent is returned; otherwise return the actual trivalue.
+ */
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised unless missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo buf)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * Found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /*
+ * Validate that we got an array if the format string specified one.
+ * And finally print out the data
+ */
+ if (is_array)
+ expand_jsonb_array(buf, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(buf, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and may contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * Expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal.
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* Easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar = dqnextchar % (sizeof(dqsuffixes) - 1);
+ }
+ /* Add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity.
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo buf, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(buf, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(buf, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(buf, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(buf, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(buf, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(buf, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(buf, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(buf, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ bool first = true;
+ StringInfoData arrayelem;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if (!JsonContainerIsArray(container))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ initStringInfo(&arrayelem);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ if (type == WJB_ELEM)
+ {
+ resetStringInfo(&arrayelem);
+
+ if (expand_one_jsonb_element(&arrayelem, param, &v, specifier, NULL))
+ {
+ if (!first)
+ appendStringInfoString(buf, arraysep);
+
+ appendBinaryStringInfo(buf, arrayelem.data, arrayelem.len);
+ first = false;
+ }
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+/*
+ * Workhorse for ddl_deparse_expand_command.
+ */
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo buf = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(buf);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, buf);
+
+ return buf->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable.
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index ddf219b..c9a0954 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1699,6 +1699,39 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ /* Open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2de0eba..ee48470 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5189,11 +5189,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
- context);
+ address = ATExecAttachPartition(wqueue, rel,
+ (PartitionCmd *) cmd->def,
+ context);
else
- ATExecAttachPartitionIdx(wqueue, rel,
- ((PartitionCmd *) cmd->def)->name);
+ address = ATExecAttachPartitionIdx(wqueue, rel,
+ ((PartitionCmd *) cmd->def)->name);
break;
case AT_DetachPartition:
cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode,
@@ -5201,12 +5202,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
Assert(cmd != NULL);
/* ATPrepCmd ensures it must be a table */
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- ATExecDetachPartition(wqueue, tab, rel,
- ((PartitionCmd *) cmd->def)->name,
- ((PartitionCmd *) cmd->def)->concurrent);
+ address = ATExecDetachPartition(wqueue, tab, rel,
+ ((PartitionCmd *) cmd->def)->name,
+ ((PartitionCmd *) cmd->def)->concurrent);
break;
case AT_DetachPartitionFinalize:
- ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
+ address = ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
diff --git a/src/backend/utils/adt/format_type.c b/src/backend/utils/adt/format_type.c
index 060fd7e..6af2257 100644
--- a/src/backend/utils/adt/format_type.c
+++ b/src/backend/utils/adt/format_type.c
@@ -27,7 +27,6 @@
#include "utils/numeric.h"
#include "utils/syscache.h"
-static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
/*
@@ -367,7 +366,7 @@ format_type_with_typemod(Oid type_oid, int32 typemod)
/*
* Add typmod decoration to the basic type name
*/
-static char *
+char *
printTypmod(const char *typname, int32 typmod, Oid typmodout)
{
char *res;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 26cf4fa..332943c 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -487,8 +487,6 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
@@ -502,7 +500,6 @@ static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1890,6 +1887,13 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+/* Internal version that reports the full partition key definition */
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2143,6 +2147,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -12017,7 +12030,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12797,7 +12810,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e41f4d..e38230d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11886,4 +11886,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'deparse the DDL command into json format string',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index 9da2300..2875106 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..bdfdf47
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.h
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/ddl_deparse.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd, bool verbose_mode);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior);
+
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 221c3e6..85a62a1 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid, int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
extern int32 type_maximum_size(Oid type_oid, int32 typemod);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..442d1ed 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -26,9 +26,11 @@ extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -41,6 +43,9 @@ extern List *select_rtable_names_for_explain(List *rtable,
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c7f550e..50911f1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1621,6 +1621,9 @@ OSInfo
OSSLCipher
OSSLDigest
OVERLAPPED
+ObjElem
+ObjTree
+ObjType
ObjectAccessDrop
ObjectAccessNamespaceSearch
ObjectAccessPostAlter
@@ -3182,6 +3185,7 @@ colormaprange
compare_context
config_var_value
contain_aggs_of_level_context
+convSpecifier
convert_testexpr_context
copy_data_source_cb
core_YYSTYPE
--
2.7.2.windows.1
[application/octet-stream] v12-0002-Support-DDL-replication.patch (123.2K, ../../OS0PR01MB571642FDCB049FC00EBDA5EC94869@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v12-0002-Support-DDL-replication.patch)
download | inline diff:
From dc29168680bfef9e224eba32d301836c559e3105 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 12 Jul 2022 15:07:47 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 213 ++++++++++++-
src/backend/commands/publicationcmds.c | 100 ++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 48 +++
src/backend/replication/logical/reorderbuffer.c | 136 +++++++++
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
31 files changed, 1578 insertions(+), 213 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index c365de3..989c49a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index f46f864..3c8da6c 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1539,6 +1542,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1572,7 +1576,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1592,6 +1596,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2179,3 +2184,209 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged
+ * tables can also be created and altered on the subscriber side. This
+ * makes it possible to directly replay the SET LOGGED command and the
+ * incoming rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+ DeparsedCommandType cmdtype;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ cmdtype = DCT_TableDropEnd;
+ else if (strncmp(obj->objecttype, "sequence", 5) == 0 ||
+ strncmp(obj->objecttype, "schema", 5) == 0 ||
+ strncmp(obj->objecttype, "index", 5) == 0)
+ cmdtype = DCT_ObjectDrop;
+ else
+ continue;
+
+ command = deparse_drop_command(obj->objidentity, obj->objecttype,
+ stmt->behavior);
+
+ if (command == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, cmdtype,
+ command, strlen(command) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..12d4b2e 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,49 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1119,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ee48470..ddcce3f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5219,7 +5219,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index aa2427b..255b0eb 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..f354062 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 8da5f90..4259f6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4299,6 +4408,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 38e3b1c..2c667bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 2cbca4a..916318f 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1974,7 +2148,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2032,6 +2207,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2145,6 +2321,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2330,6 +2507,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index f502df9..59c30d0 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c871cb7..ae7ca5e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3866,6 +3866,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3881,23 +3882,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3913,6 +3920,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3936,6 +3944,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4015,6 +4025,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 88d92a0..8b9c9a5 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6047,7 +6047,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6082,6 +6082,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6169,6 +6173,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6185,6 +6190,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6198,6 +6204,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6246,6 +6255,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6260,6 +6271,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6270,6 +6283,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e38230d..571b9b6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11892,4 +11892,13 @@
{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index 4a01f87..de58035 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v12-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (15.0K, ../../OS0PR01MB571642FDCB049FC00EBDA5EC94869@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v12-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From dd86acbf56e329cb0be653ac2dede08012b44631 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Wed, 29 Jun 2022 17:04:32 +0800
Subject: [PATCH] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 24 +++++
src/backend/commands/event_trigger.c | 172 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 232 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index cbaa146..aceb815 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -3409,6 +3409,27 @@ deparse_AlterTableStmt(CollectedCommand *cmd)
}
/*
+ * Deparse CREATE TABLE AS command.
+ *
+ * deparse_CreateStmt do the actual work as we deparse the final CreateStmt for
+ * CREATE TABLE AS command.
+ */
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
+/*
* Handle deparsing of simple commands.
*
* This function should cover all cases handled in ProcessUtilitySlow.
@@ -3545,6 +3566,9 @@ deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index bd6a359..ed6cc63 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -585,7 +587,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -868,6 +871,163 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ /* Do nothing if no command was collected. */
+ if (!currentEventTriggerState->currentCommand)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1148,7 +1308,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1872,6 +2033,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1889,6 +2051,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 12d4b2e..ff75999 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -966,6 +966,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -977,6 +982,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6b0a865..490b73b 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index f7f7165..7fb8cb2 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 571b9b6..a4adc9a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11901,4 +11901,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-07-20 12:28 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 2 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-07-20 12:28 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>; Peter Smith <smithpb2250@gmail.com>
On Tuesday, July 12, 2022 5:04 PM houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com> wrote:
> On Friday, July 8, 2022 10:26 AM Peter Smith <smithpb2250@gmail.com>
> wrote:
>
>
> > Here are some review comments for the patch v11-0001:
>
> Attach the V12 patch set which include the following changes:
>
> * Address comments from peter[1]
> * Refactor the deparser and provide an option to control whether output the
> "not present" syntax part.
> And for DDL replication, we don't WAL log the "not present" syntax string for
> now.
> * Address most comments from Vignesh[2] except the one about pg_dump
> vs the event trigger for ddl replication which need some more research.
Attach the new version patch set which added support for CREATE/ALTER FUNCTION,
CREATE/ALTER PROCEDURE, CREATE/ALTER TRIGGER, RENAME TABLE/INDEX/SEQUENCE.
Thanks to Vignesh for helping off list.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v13-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (15.0K, ../../OS0PR01MB5716FCCEF8009E4464042476948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v13-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From dd86acbf56e329cb0be653ac2dede08012b44631 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Wed, 29 Jun 2022 17:04:32 +0800
Subject: [PATCH] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 24 +++++
src/backend/commands/event_trigger.c | 172 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 232 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index cbaa146..aceb815 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -3409,6 +3409,27 @@ deparse_AlterTableStmt(CollectedCommand *cmd)
}
/*
+ * Deparse CREATE TABLE AS command.
+ *
+ * deparse_CreateStmt do the actual work as we deparse the final CreateStmt for
+ * CREATE TABLE AS command.
+ */
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
+/*
* Handle deparsing of simple commands.
*
* This function should cover all cases handled in ProcessUtilitySlow.
@@ -3545,6 +3566,9 @@ deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
default:
elog(ERROR, "unexpected deparse node type %d", cmd->type);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index bd6a359..ed6cc63 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -585,7 +587,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -868,6 +871,163 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ /* Do nothing if no command was collected. */
+ if (!currentEventTriggerState->currentCommand)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1148,7 +1308,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1872,6 +2033,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1889,6 +2051,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 12d4b2e..ff75999 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -966,6 +966,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -977,6 +982,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6b0a865..490b73b 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index f7f7165..7fb8cb2 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 571b9b6..a4adc9a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11901,4 +11901,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v13-0002-Support-DDL-replication.patch (123.5K, ../../OS0PR01MB5716FCCEF8009E4464042476948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v13-0002-Support-DDL-replication.patch)
download | inline diff:
From 2d1148173c5fb45cd31e70edc171538fb67a5f5b Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 12 Jul 2022 15:07:47 +0800
Subject: [PATCH] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
- And other table unrelated command
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 214 ++++++++++++-
src/backend/commands/publicationcmds.c | 106 +++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 48 +++
src/backend/replication/logical/reorderbuffer.c | 136 +++++++++
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
31 files changed, 1585 insertions(+), 213 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index c365de3..989c49a 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index f46f864..db653a3 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1539,6 +1542,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1572,7 +1576,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1592,6 +1596,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2179,3 +2184,210 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged
+ * tables can also be created and altered on the subscriber side. This
+ * makes it possible to directly replay the SET LOGGED command and the
+ * incoming rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist = RELPERSISTENCE_PERMANENT;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ if (get_rel_relkind(relid))
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+ DeparsedCommandType cmdtype;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ cmdtype = DCT_TableDropEnd;
+ else if (strncmp(obj->objecttype, "sequence", 5) == 0 ||
+ strncmp(obj->objecttype, "schema", 5) == 0 ||
+ strncmp(obj->objecttype, "index", 5) == 0)
+ cmdtype = DCT_ObjectDrop;
+ else
+ continue;
+
+ command = deparse_drop_command(obj->objidentity, obj->objecttype,
+ stmt->behavior);
+
+ if (command == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, cmdtype,
+ command, strlen(command) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 8e64574..326ec61 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,55 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_CREATE_PROCEDURE,
+ CMDTAG_ALTER_PROCEDURE,
+ CMDTAG_CREATE_FUNCTION,
+ CMDTAG_ALTER_FUNCTION,
+ CMDTAG_CREATE_TRIGGER,
+ CMDTAG_ALTER_TRIGGER,
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1125,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 39d8d1e..41069b5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5219,7 +5219,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index c5c6a2b..e4e0b62 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..f354062 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 88a37fd..1f6d4ea 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4299,6 +4408,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 38e3b1c..2c667bb 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 2cbca4a..916318f 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -54,6 +54,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -407,6 +414,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -480,6 +488,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1362,9 +1371,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1398,6 +1420,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1427,8 +1456,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1448,8 +1477,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1579,6 +1619,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1656,8 +1699,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1697,6 +1740,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Currently we always forward.
*/
@@ -1974,7 +2148,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2032,6 +2207,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2145,6 +2321,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2330,6 +2507,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index bdb771d..6c10974 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index e4fdb6b..3189a4f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3873,6 +3873,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3888,23 +3889,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3920,6 +3927,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3943,6 +3951,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4022,6 +4032,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1d21c29..69b94ae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 88d92a0..8b9c9a5 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6047,7 +6047,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6082,6 +6082,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6169,6 +6173,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6185,6 +6190,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6198,6 +6204,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6246,6 +6255,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6260,6 +6271,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6270,6 +6283,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e38230d..571b9b6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11892,4 +11892,13 @@
{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index 539dc8e..933d297 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -89,6 +89,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -200,6 +212,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -219,6 +244,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -237,6 +263,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index eafedd6..8ebcc12 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index d109d0b..737284c 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 60acbd1..d534cbf 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5969,9 +5969,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 274b37d..09b14e0 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v13-0001-Functions-to-deparse-DDL-commands.patch (166.6K, ../../OS0PR01MB5716FCCEF8009E4464042476948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v13-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From f6c0ba0360954c8aba08ef1231a0dd7b5b631fe7 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides a base for logical replication of DDL statements. Currently,
the patch has support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/ALTER/DROP SCHEMA
CREATE/ALTER FUNCTION/PROCEDURE
CREATE/ALTER TRIGGER
CREATE/DROP INDEX
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 4523 +++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 760 ++++++
src/backend/commands/sequence.c | 33 +
src/backend/commands/tablecmds.c | 17 +-
src/backend/utils/adt/format_type.c | 3 +-
src/backend/utils/adt/ruleutils.c | 149 +-
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 23 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/builtins.h | 1 +
src/include/utils/ruleutils.h | 13 +
src/tools/pgindent/typedefs.list | 4 +
14 files changed, 5458 insertions(+), 78 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..0c0219d
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,4523 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_language.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "funcapi.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/guc.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+/* Estimated length of the generated jsonb string */
+#define JSONB_ESTIMATED_LEN 128
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+ StringInfo fmtinfo;
+ bool present;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+bool verbose = true;
+
+static ObjElem *new_object(ObjType type, char *name);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static void append_float_object(ObjTree *tree, char *name, float8 value);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+static List *FunctionGetDefaults(text *proargdefaults);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_extended, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
+
+ /*
+ * Special-case crock for types with strange typmod rules where we put
+ * typmod at the middle of name(e.g. TIME(6) with time zone ). We cannot
+ * schema-qualify nor add quotes to the type name in these cases.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmod. */
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
+ break;
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+ }
+ else
+ {
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+ }
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Return the defaults values of arguments to a function, as a list of
+ * deparsed expressions.
+ */
+static List *
+FunctionGetDefaults(text *proargdefaults)
+{
+ List *nodedefs;
+ List *strdefs = NIL;
+ ListCell *cell;
+
+ nodedefs = (List *) stringToNode(TextDatumGetCString(proargdefaults));
+ if (!IsA(nodedefs, List))
+ elog(ERROR, "proargdefaults is not a list");
+
+ foreach(cell, nodedefs)
+ {
+ Node *onedef = lfirst(cell);
+
+ strdefs = lappend(strdefs, deparse_expression(onedef, NIL, false, false));
+ }
+
+ return strdefs;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(char *fmt)
+{
+ ObjTree *params;
+
+ params = palloc0(sizeof(ObjTree));
+ params->present = true;
+ slist_init(¶ms->params);
+
+ if (fmt)
+ {
+ params->fmtinfo = makeStringInfo();
+ appendStringInfoString(params->fmtinfo, fmt);
+ }
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree(fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+ elem = new_object(type, NULL);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ break;
+ case ObjTypeBool:
+ elem->value.boolean = va_arg(args, int);
+ break;
+ case ObjTypeString:
+ elem->value.string = va_arg(args, char *);
+ break;
+ case ObjTypeArray:
+ elem->value.array = va_arg(args, List *);
+ break;
+ case ObjTypeInteger:
+ elem->value.integer = va_arg(args, int64);
+ break;
+ case ObjTypeFloat:
+ elem->value.flt = va_arg(args, double);
+ break;
+ case ObjTypeObject:
+ elem->value.object = va_arg(args, ObjTree *);
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/*
+ * Append the input format string to the ObjTree.
+ */
+static void
+append_format_string(ObjTree *tree, char *sub_fmt)
+{
+ int len;
+ char *fmt;
+
+ if (!tree->fmtinfo)
+ return;
+
+ fmt = tree->fmtinfo->data;
+ len = tree->fmtinfo->len;
+
+ /* Add a separator if necessary */
+ if (len > 0 && fmt[len - 1] != ' ')
+ appendStringInfoSpaces(tree->fmtinfo, 1);
+
+ appendStringInfoString(tree->fmtinfo, sub_fmt);
+}
+
+/*
+ * Return the object name which is extracted from the input "*%{name[:.]}*"
+ * style string. And append the input format string to the ObjTree.
+ */
+static char *
+append_object_to_format_string(ObjTree *tree, char *sub_fmt)
+{
+ StringInfoData object_name;
+ const char *end_ptr;
+ const char *cp;
+ bool start_copy = false;
+
+ if (sub_fmt == NULL || tree->fmtinfo == NULL)
+ return sub_fmt;
+
+ initStringInfo(&object_name);
+ end_ptr = sub_fmt + strlen(sub_fmt);
+
+ for (cp = sub_fmt; cp < end_ptr; cp++)
+ {
+ if (*cp == '{')
+ {
+ start_copy = true;
+ continue;
+ }
+
+ if (!start_copy)
+ continue;
+
+ if (*cp == ':' || *cp == '}')
+ break;
+
+ appendStringInfoCharMacro(&object_name, *cp);
+ }
+
+ if (object_name.len == 0)
+ elog(ERROR, "object name not found");
+
+ append_format_string(tree, sub_fmt);
+
+ return object_name.data;
+}
+
+/* Allocate a new parameter */
+static ObjElem *
+new_object(ObjType type, char *name)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = name;
+ param->objtype = type;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *sub_fmt)
+{
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ append_premade_object(tree, new_object(ObjTypeNull, object_name));
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *sub_fmt, bool value)
+{
+ ObjElem *param;
+ char *object_name = sub_fmt;
+ bool is_present_flag = false;
+
+ Assert(sub_fmt);
+
+ if (strcmp(sub_fmt, "present") == 0)
+ {
+ is_present_flag = true;
+ tree->present = value;
+ }
+
+ if (!verbose && !tree->present)
+ return;
+
+ if (!is_present_flag)
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeBool, object_name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = new_object(ObjTypeString, NULL);
+ param->value.string = value;
+
+ return param;
+}
+
+/* Append a string parameter to a tree */
+static void
+append_string_object(ObjTree *tree, char *sub_fmt, char *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && (value == NULL || value[0] == '\0'))
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeString, object_name);
+ param->value.string = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeObject, NULL);
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *sub_fmt, ObjTree *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && !value->present)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeObject, object_name);
+ param->value.object = value;
+ append_premade_object(tree, param);
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *sub_fmt, List *array)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && list_length(array) == 0)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeArray, object_name);
+ param->value.array = array;
+ append_premade_object(tree, param);
+}
+
+/*
+ * Append a float8 parameter to a tree.
+ */
+static void
+append_float_object(ObjTree *tree, char *sub_fmt, float8 value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeFloat, object_name);
+ param->value.flt = value;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Process the pre-built format string from the ObjTree into the output parse
+ * state.
+ */
+static void
+objtree_fmt_to_jsonb_element(JsonbParseState *state, ObjTree *tree)
+{
+ JsonbValue key;
+ JsonbValue val;
+
+ if (!tree->fmtinfo)
+ return;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.val = "fmt";
+ key.val.string.len = strlen(key.val.string.val);
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the pre-built format string */
+ val.type = jbvString;
+ val.val.string.len = tree->fmtinfo->len;
+ val.val.string.val = tree->fmtinfo->data;
+ pushJsonbValue(&state, WJB_VALUE, &val);
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb.
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ objtree_fmt_to_jsonb_element(state, tree);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (OidIsValid(typnspid))
+ typnsp = get_namespace_name_or_temp(typnspid);
+ else
+ typnsp = pstrdup("");
+
+ typeParam = new_objtree_VA(NULL, 4,
+ "schemaname", ObjTypeString, typnsp,
+ "typename", ObjTypeString, typename,
+ "typmod", ObjTypeString, typmodstr,
+ "typarray", ObjTypeBool, typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements.
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+
+ qualified = new_objtree_VA(NULL, 2,
+ "schemaname", ObjTypeString, namespace,
+ "objname", ObjTypeString, pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId.
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value.
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * Deparse the definition of column identity.
+ */
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *column;
+ ObjTree *identobj;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+ char *identfmt;
+ char *objfmt;
+
+ column = new_objtree("");
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(column, "present", false);
+ return column;
+ }
+
+ if (alter_table)
+ {
+ identfmt = "SET GENERATED ";
+ objfmt = "%{option}s";
+ }
+ else
+ {
+ identfmt = "GENERATED ";
+ objfmt = "%{option}s AS IDENTITY";
+ }
+
+ identobj = new_objtree(identfmt);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(identobj, objfmt, "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(identobj, objfmt, "BY DEFAULT");
+ else
+ append_bool_object(identobj, "present", false);
+
+ append_object_object(column, "%{identity_type}s", identobj);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* Definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* We purposefully do not emit OWNED BY here */
+
+ if (alter_table)
+ append_array_object(column, "%{seq_definition: }s", elems);
+ else
+ append_array_object(column, "( %{seq_definition: }s )", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return column;
+}
+
+/*
+ * Deparse a CreateTrigStmt (CREATE TRIGGER)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateTrigStmt(Oid objectId, Node *parsetree)
+{
+ CreateTrigStmt *node = (CreateTrigStmt *) parsetree;
+ Relation pg_trigger;
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ ObjTree *trigger;
+ ObjTree *tmp;
+ int tgnargs;
+ List *list;
+ List *events;
+
+ pg_trigger = table_open(TriggerRelationId, AccessShareLock);
+
+ trigTup = get_catalog_object_by_oid(pg_trigger, Anum_pg_trigger_oid, objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{constraint}s TRIGGER %{name}I %{time}s %{events: OR }s
+ * ON %{relation}D %{from_table}s %{constraint_attrs: }s
+ * FOR EACH %{for_each}s %{when}s EXECUTE PROCEDURE %{function}s
+ */
+ trigger = new_objtree("CREATE");
+
+ append_string_object(trigger, "%{constraint}s",
+ node->isconstraint ? "CONSTRAINT" : "");
+
+ append_string_object(trigger, "TRIGGER %{name}I", node->trigname);
+
+ if (node->timing == TRIGGER_TYPE_BEFORE)
+ append_string_object(trigger, "%{time}s", "BEFORE");
+ else if (node->timing == TRIGGER_TYPE_AFTER)
+ append_string_object(trigger, "%{time}s", "AFTER");
+ else if (node->timing == TRIGGER_TYPE_INSTEAD)
+ append_string_object(trigger, "%{time}s", "INSTEAD OF");
+ else
+ elog(ERROR, "unrecognized trigger timing value %d", node->timing);
+
+ /*
+ * Decode the events that the trigger fires for. The output is a list;
+ * in most cases it will just be a string with the even name, but when
+ * there's an UPDATE with a list of columns, we return a JSON object.
+ */
+ events = NIL;
+ if (node->events & TRIGGER_TYPE_INSERT)
+ events = lappend(events, new_string_object("INSERT"));
+ if (node->events & TRIGGER_TYPE_DELETE)
+ events = lappend(events, new_string_object("DELETE"));
+ if (node->events & TRIGGER_TYPE_TRUNCATE)
+ events = lappend(events, new_string_object("TRUNCATE"));
+ if (node->events & TRIGGER_TYPE_UPDATE)
+ {
+ if (node->columns == NIL)
+ {
+ events = lappend(events, new_string_object("UPDATE"));
+ }
+ else
+ {
+ ObjTree *update;
+ ListCell *cell;
+ List *cols = NIL;
+
+ /*
+ * Currently only UPDATE OF can be objects in the output JSON, but
+ * we add a "kind" element so that user code can distinguish
+ * possible future new event types.
+ */
+ update = new_objtree_VA("UPDATE OF", 1,
+ "kind", ObjTypeString, "update_of");
+
+ foreach(cell, node->columns)
+ {
+ char *colname = strVal(lfirst(cell));
+
+ cols = lappend(cols,
+ new_string_object(colname));
+ }
+
+ append_array_object(update, "%{columns:, }I", cols);
+
+ events = lappend(events,
+ new_object_object(update));
+ }
+ }
+ append_array_object(trigger, "%{events: OR }s", events);
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid);
+ append_object_object(trigger, "ON %{relation}D", tmp);
+
+ tmp = new_objtree_VA("FROM", 0);
+ if (trigForm->tgconstrrelid)
+ {
+ ObjTree *rel;
+
+ rel = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgconstrrelid);
+ append_object_object(tmp, "%{relation}D", rel);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{from_table}s", tmp);
+
+ list = NIL;
+ if (node->deferrable)
+ list = lappend(list,
+ new_string_object("DEFERRABLE"));
+ if (node->initdeferred)
+ list = lappend(list,
+ new_string_object("INITIALLY DEFERRED"));
+ append_array_object(trigger, "%{constraint_attrs: }s", list);
+
+ append_string_object(trigger, "FOR EACH %{for_each}s",
+ node->row ? "ROW" : "STATEMENT");
+
+ tmp = new_objtree_VA("WHEN", 0);
+ if (node->whenClause)
+ {
+ Node *whenClause;
+ Datum value;
+ bool isnull;
+
+ value = fastgetattr(trigTup, Anum_pg_trigger_tgqual,
+ RelationGetDescr(pg_trigger), &isnull);
+ if (isnull)
+ elog(ERROR, "bogus NULL tgqual");
+
+ whenClause = stringToNode(TextDatumGetCString(value));
+ append_string_object(tmp, "(%{clause}s)",
+ pg_get_trigger_whenclause(trigForm,
+ whenClause,
+ false));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{when}s", tmp);
+
+ tmp = new_objtree_VA("%{funcname}D", 1,
+ "funcname", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ trigForm->tgfoid));
+ list = NIL;
+ tgnargs = trigForm->tgnargs;
+ if (tgnargs > 0)
+ {
+ bytea *tgargs;
+ char *argstr;
+ bool isnull;
+ int findx;
+ int lentgargs;
+ char *p;
+
+ tgargs = DatumGetByteaP(fastgetattr(trigTup,
+ Anum_pg_trigger_tgargs,
+ RelationGetDescr(pg_trigger),
+ &isnull));
+ if (isnull)
+ elog(ERROR, "invalid NULL tgargs");
+ argstr = (char *) VARDATA(tgargs);
+ lentgargs = VARSIZE_ANY_EXHDR(tgargs);
+
+ p = argstr;
+ for (findx = 0; findx < tgnargs; findx++)
+ {
+ size_t tlen;
+
+ /* verify that the argument encoding is correct */
+ tlen = strlen(p);
+ if (p + tlen >= argstr + lentgargs)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid argument string (%s) for trigger \"%s\"",
+ argstr, NameStr(trigForm->tgname))));
+
+ list = lappend(list, new_string_object(p));
+
+ p += tlen + 1;
+ }
+ }
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{args:, }L", list); /* might be NIL */
+ append_format_string(tmp, ")");
+
+ append_object_object(trigger, "EXECUTE PROCEDURE %{function}s", tmp);
+
+ table_close(pg_trigger, AccessShareLock);
+
+ return trigger;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.).
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted.
+ *
+ * XXX maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I %{coltype}T %{compression}s %{default}s %{not_null}s
+ * %{collation}s
+ */
+ column = new_objtree_VA("%{name}I %{coltype}T", 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree("COMPRESSION");
+ if (coldef->compression)
+ append_string_object(tmp, "%{compression_method}I", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "%{compression_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "%{compression}s", tmp);
+
+ tmp = new_objtree("COLLATE");
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{collation}s", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "%{identity_column}s", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree("GENERATED ALWAYS AS");
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "%{generation_expr}s STORED", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "%{generated_column}s", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE OF deparsing.
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I WITH OPTIONS %{default}s %{not_null}s.
+ */
+ column = new_objtree_VA("%{name}I WITH OPTIONS", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /*
+ * Generated columns are not supported on typed tables, so we are done.
+ */
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparse_TableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing.
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *constr;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ constr = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(constr));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * Deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *oncommit;
+
+ oncommit = new_objtree("ON COMMIT");
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(oncommit, "%{on_commit_value}s", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(oncommit, "%{on_commit_value}s");
+ append_bool_object(oncommit, "present", false);
+ break;
+ }
+
+ return oncommit;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ set = new_objtree("");
+ optname = new_objtree("");
+
+ if (elem->defnamespace != NULL)
+ append_string_object(optname, "%{schema}I.", elem->defnamespace);
+
+ append_string_object(optname, "%{label}I", elem->defname);
+
+ append_object_object(set, "%{label}s", optname);
+
+ if (!is_reset)
+ append_string_object(set, " = %{value}L",
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *colset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER COLUMN %{column}I RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "ALTER COLUMN %{column}I RESET ";
+ else
+ fmt = "ALTER COLUMN %{column}I SET ";
+
+ colset = new_objtree_VA(fmt, 1, "column", ObjTypeString, subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(colset, "(%{options:, }s)", sets);
+
+ return colset;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *relset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "RESET ";
+ else
+ fmt = "SET ";
+
+ relset = new_objtree(fmt);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(relset, "(%{options:, }s)", sets);
+
+ return relset;
+}
+
+/*
+ * Deparse the inherits relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+/*
+ * Obtain the deparsed partition bound expression for the given table.
+ */
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* Output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, keyno == 0 ? "" : ", ");
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* Expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* If it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* Output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* Output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* Report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+}
+
+/*
+ * Deparse an IndexStmt.
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I ON
+ * %{table}D USING %{index_am}s (%{definition}s) %{with}s %{tablespace}s
+ * %{where_clause}s
+ */
+ indexStmt = new_objtree("CREATE");
+
+ append_string_object(indexStmt, "%{unique}s",
+ node->unique ? "UNIQUE" : "");
+
+ append_format_string(indexStmt, "INDEX");
+
+ append_string_object(indexStmt, "%{concurrently}s",
+ node->concurrent ? "CONCURRENTLY" : "");
+
+ append_string_object(indexStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(indexStmt, "%{name}I",
+ RelationGetRelationName(idxrel));
+
+ append_object_object(indexStmt, "ON %{table}D",
+ new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel)));
+
+ append_string_object(indexStmt, "USING %{index_am}s", index_am);
+
+ append_string_object(indexStmt, "(%{definition}s)", definition);
+
+ /* reloptions */
+ tmp = new_objtree("WITH");
+ if (reloptions)
+ append_string_object(tmp, "(%{opts}s)", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{with}s", tmp);
+
+ /* tablespace */
+ tmp = new_objtree("TABLESPACE");
+ if (tablespace)
+ append_string_object(tmp, "%{tablespace}s", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{tablespace}s", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree("WHERE");
+ if (whereClause)
+ append_string_object(tmp, "%{where}s", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{where_clause}s", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * Deparse a CreateStmt (CREATE TABLE).
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D [OF
+ * %{of_type}T | PARTITION OF %{parent_identity}D] %{table_elements}s
+ * %{inherits}s %{partition_by}s %{access_method}s %{with_clause}s
+ * %{on_commit}s %{tablespace}s
+ */
+ createStmt = new_objtree("CREATE");
+
+ append_string_object(createStmt, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ append_format_string(createStmt, "TABLE");
+
+ append_string_object(createStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "%{identity}D", tmp);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ /*
+ * Typed tables and partitions use a slightly different format string: we
+ * must not put table_elements with parents directly in the fmt string,
+ * because if there are no options the parens must not be emitted; and
+ * also, typed tables do not allow for inheritance.
+ */
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "OF %{of_type}T", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_object_object(createStmt, "%{parent_identity}D",
+ elem->value.object);
+ }
+
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ {
+ tmp = new_objtree("");
+ append_bool_object(tmp, "present", false);
+ }
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "(%{table_elements:, }s)", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "(%{table_elements:, }s)", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree("INHERITS");
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "(%{parents:, }D)", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "(%{parents:, }D)");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{inherits}s", tmp);
+ }
+
+ tmp = new_objtree("TABLESPACE");
+ if (node->tablespacename)
+ append_string_object(tmp, "%{tablespace}I", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "%{tablespace}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{tablespace}s", tmp);
+
+ append_object_object(createStmt, "%{on_commit}s",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "%{partition_bound}s",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree("PARTITION BY");
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "%{definition}s", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "%{definition}s");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{partition_by}s", tmp);
+
+ /* USING clause */
+ tmp = new_objtree("USING");
+ if (node->accessMethod)
+ append_string_object(tmp, "%{access_method}I", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "%{access_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{access_method}s", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree("WITH");
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "(%{with:, }s)", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "%{with_clause}s", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_FunctionSet(VariableSetKind kind, char *name, char *value)
+{
+ ObjTree *r;
+
+ if (kind == VAR_RESET_ALL)
+ {
+ r = new_objtree("RESET ALL");
+ }
+ else if (value != NULL)
+ {
+ r = new_objtree_VA("SET %{set_name}I", 1,
+ "set_name", ObjTypeString, name);
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be
+ * quoted.
+ */
+ if (GetConfigOptionFlags(name, true) & GUC_LIST_QUOTE)
+ append_string_object(r, "TO %{set_value}s", value);
+ else
+ append_string_object(r, "TO %{set_value}L", value);
+ }
+ else
+ {
+ r = new_objtree("RESET");
+ append_string_object(r, "%{set_name}I", name);
+ }
+
+ return r;
+}
+
+/*
+ * deparse_CreateFunctionStmt
+ * Deparse a CreateFunctionStmt (CREATE FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
+static ObjTree *
+deparse_CreateFunction(Oid objectId, Node *parsetree)
+{
+ CreateFunctionStmt *node = (CreateFunctionStmt *) parsetree;
+ ObjTree *createFunc;
+ ObjTree *tmp;
+ Datum tmpdatum;
+ char *source;
+ char *probin;
+ List *params;
+ List *defaults;
+ List *sets = NIL;
+ List *types = NIL;
+ ListCell *cell;
+ ListCell *curdef;
+ ListCell *table_params = NULL;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ HeapTuple langTup;
+ Oid *typarray;
+ Oid *trftypes;
+ Form_pg_language langForm;
+ int i;
+ int typnum;
+ int ntypes;
+ bool isnull;
+ bool isfunction;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /* get the corresponding pg_language tuple */
+ langTup = SearchSysCache1(LANGOID, procForm->prolang);
+ if (!HeapTupleIsValid(langTup))
+ elog(ERROR, "cache lookup failure for language with OID %u",
+ procForm->prolang);
+ langForm = (Form_pg_language) GETSTRUCT(langTup);
+
+ /*
+ * Determine useful values for prosrc and probin. We cope with probin
+ * being either NULL or "-", but prosrc must have a valid value.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_prosrc, &isnull);
+ if (isnull)
+ elog(ERROR, "null prosrc in function with OID %u", objectId);
+ source = TextDatumGetCString(tmpdatum);
+
+ /* Determine a useful value for probin */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_probin, &isnull);
+ if (isnull)
+ probin = NULL;
+ else
+ {
+ probin = TextDatumGetCString(tmpdatum);
+ if (probin[0] == '\0' || strcmp(probin, "-") == 0)
+ probin = NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{or_replace}s FUNCTION %{signature}s RETURNS %{return_type}s
+ * LANGUAGE %{transform_type}s %{language}I %{window}s %{volatility}s
+ * %{parallel_safety}s %{leakproof}s %{strict}s %{security_definer}s
+ * %{cost}s %{rows}s %{support}s %{set_options: }s AS %{objfile}L,
+ * %{symbol}L
+ */
+ createFunc = new_objtree("CREATE");
+
+ append_string_object(createFunc, "%{or_replace}s",
+ node->replace ? "OR REPLACE" : "");
+
+ /*
+ * To construct the arguments array, extract the type OIDs from the
+ * function's pg_proc entry. If pronargs equals the parameter list length,
+ * there are no OUT parameters and thus we can extract the type OID from
+ * proargtypes; otherwise we need to decode proallargtypes, which is
+ * a bit more involved.
+ */
+ typarray = palloc(list_length(node->parameters) * sizeof(Oid));
+ if (list_length(node->parameters) > procForm->pronargs)
+ {
+ bool isnull;
+ Datum alltypes;
+ Datum *values;
+ bool *nulls;
+ int nelems;
+
+ alltypes = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proallargtypes, &isnull);
+ if (isnull)
+ elog(ERROR, "NULL proallargtypes, but more parameters than args");
+ deconstruct_array(DatumGetArrayTypeP(alltypes),
+ OIDOID, 4, 't', 'i',
+ &values, &nulls, &nelems);
+ if (nelems != list_length(node->parameters))
+ elog(ERROR, "mismatched proallargatypes");
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = values[i];
+ }
+ else
+ {
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = procForm->proargtypes.values[i];
+ }
+
+ /*
+ * If there are any default expressions, we read the deparsed expression as
+ * a list so that we can attach them to each argument.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proargdefaults, &isnull);
+ if (!isnull)
+ {
+ defaults = FunctionGetDefaults(DatumGetTextP(tmpdatum));
+ curdef = list_head(defaults);
+ }
+ else
+ {
+ defaults = NIL;
+ curdef = NULL;
+ }
+
+ /*
+ * Now iterate over each parameter in the parsetree to create the
+ * parameters array.
+ */
+ params = NIL;
+ typnum = 0;
+ foreach(cell, node->parameters)
+ {
+ FunctionParameter *param = (FunctionParameter *) lfirst(cell);
+ ObjTree *paramobj;
+ ObjTree *defaultval;
+ ObjTree *name;
+
+ /*
+ * A PARAM_TABLE parameter indicates end of input arguments; the
+ * following parameters are part of the return type. We ignore them
+ * here, but keep track of the current position in the list so that
+ * we can easily produce the return type below.
+ */
+ if (param->mode == FUNC_PARAM_TABLE)
+ {
+ table_params = cell;
+ break;
+ }
+
+ /*
+ * Verbose syntax for paramater: %{mode}s %{name}s %{type}T %{default}s
+ *
+ * Note that %{name}s is a string here, not an identifier; the reason
+ * for this is that an absent parameter name must produce an empty
+ * string, not "", which is what would happen if we were to use
+ * %{name}I here. So we add another level of indirection to allow us
+ * to inject a "present" parameter.
+ */
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{mode}s",
+ param->mode == FUNC_PARAM_IN ? "IN" :
+ param->mode == FUNC_PARAM_OUT ? "OUT" :
+ param->mode == FUNC_PARAM_INOUT ? "INOUT" :
+ param->mode == FUNC_PARAM_VARIADIC ? "VARIADIC" :
+ "IN");
+
+ /* optional wholesale suppression of "name" occurs here */
+
+ name = new_objtree("");
+ append_string_object(name, "%{name}I",
+ param->name ? param->name : "NULL");
+
+ append_bool_object(name, "present",
+ param->name ? true : false);
+
+ append_object_object(paramobj, "%{name}s", name);
+
+ defaultval = new_objtree("DEFAULT");
+ if (PointerIsValid(param->defexpr))
+ {
+ char *expr;
+
+ if (curdef == NULL)
+ elog(ERROR, "proargdefaults list too short");
+ expr = lfirst(curdef);
+
+ append_string_object(defaultval, "%{value}s", expr);
+ curdef = lnext(defaults, curdef);
+ }
+ else
+ append_bool_object(defaultval, "present", false);
+
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+
+ append_object_object(paramobj, "%{default}s", defaultval);
+
+ params = lappend(params, new_object_object(paramobj));
+ }
+
+ tmp = new_objtree_VA("%{identity}D", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{arguments:, }s", params);
+ append_format_string(tmp, ")");
+
+ isfunction = (procForm->prokind != PROKIND_PROCEDURE);
+
+ if (isfunction)
+ append_object_object(createFunc, "FUNCTION %{signature}s", tmp);
+ else
+ append_object_object(createFunc, "PROCEDURE %{signature}s", tmp);
+
+ /*
+ * A return type can adopt one of two forms: either a [SETOF] some_type, or
+ * a TABLE(list-of-types). We can tell the second form because we saw a
+ * table param above while scanning the argument list.
+ */
+ if (table_params == NULL)
+ {
+ tmp = new_objtree_VA("", 1,
+ "return_form", ObjTypeString, "plain");
+ append_string_object(tmp, "%{setof}s",
+ procForm->proretset ? "SETOF" : "");
+ append_object_object(tmp, "%{rettype}T",
+ new_objtree_for_type(procForm->prorettype, -1));
+ }
+ else
+ {
+ List *rettypes = NIL;
+ ObjTree *paramobj;
+
+ tmp = new_objtree_VA("TABLE", 1,
+ "return_form", ObjTypeString, "table");
+ for (; table_params != NULL; table_params = lnext(node->parameters, table_params))
+ {
+ FunctionParameter *param = lfirst(table_params);
+
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{name}I", param->name);
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+ rettypes = lappend(rettypes, new_object_object(paramobj));
+ }
+
+ append_array_object(tmp, "(%{rettypes:, }s)", rettypes);
+ }
+
+ if (isfunction)
+ append_object_object(createFunc, "RETURNS %{return_type}s", tmp);
+
+ /* TRANSFORM FOR TYPE */
+ tmp = new_objtree("TRANSFORM");
+
+ ntypes = get_func_trftypes(procTup, &trftypes);
+ for (i = 0; i < ntypes; i++)
+ {
+ tmp = new_objtree_VA("FOR TYPE %{type}T", 1,
+ "type", ObjTypeObject,
+ new_objtree_for_type(trftypes[i], -1));
+ types = lappend(types, tmp);
+ }
+
+ if (types)
+ append_array_object(tmp, "%{for_type:, }s", types);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{transform_type}s", tmp);
+
+ append_string_object(createFunc, "LANGUAGE %{language}I",
+ NameStr(langForm->lanname));
+
+ if (isfunction)
+ {
+ append_string_object(createFunc, "%{window}s",
+ procForm->prokind == PROKIND_WINDOW ? "WINDOW" : "");
+ append_string_object(createFunc, "%{volatility}s",
+ procForm->provolatile == PROVOLATILE_VOLATILE ?
+ "VOLATILE" :
+ procForm->provolatile == PROVOLATILE_STABLE ?
+ "STABLE" :
+ procForm->provolatile == PROVOLATILE_IMMUTABLE ?
+ "IMMUTABLE" : "INVALID VOLATILITY");
+
+ append_string_object(createFunc, "%{parallel_safety}s",
+ procForm->proparallel == PROPARALLEL_SAFE ?
+ "PARALLEL SAFE" :
+ procForm->proparallel == PROPARALLEL_RESTRICTED ?
+ "PARALLEL RESTRICTED" : "PARALLEL UNSAFE");
+
+ append_string_object(createFunc, "%{leakproof}s",
+ procForm->proleakproof ? "LEAKPROOF" : "");
+ append_string_object(createFunc, "%{strict}s",
+ procForm->proisstrict ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT");
+
+ append_string_object(createFunc, "%{security_definer}s",
+ procForm->prosecdef ?
+ "SECURITY DEFINER" : "SECURITY INVOKER");
+
+ append_object_object(createFunc, "%{cost}s",
+ new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ procForm->procost));
+
+ tmp = new_objtree("ROWS");
+ if (procForm->prorows == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n", procForm->prorows);
+ append_object_object(createFunc, "%{rows}s", tmp);
+
+ tmp = new_objtree("SUPPORT %{name}s");
+ if (procForm->prosupport)
+ {
+ Oid argtypes[1];
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{support}s", tmp);
+ }
+
+ foreach(cell, node->options)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ sets = lappend(sets, new_object_object(tmp));
+ }
+ }
+ append_array_object(createFunc, "%{set_options: }s", sets);
+
+ /* Add the function definition */
+ (void) SysCacheGetAttr(PROCOID, procTup, Anum_pg_proc_prosqlbody, &isnull);
+ if (procForm->prolang == SQLlanguageId && !isnull)
+ {
+ StringInfoData buf;
+
+ initStringInfo(&buf);
+ print_function_sqlbody(&buf, procTup);
+
+ append_string_object(createFunc, "%{definition}s", buf.data);
+ }
+ else if (probin == NULL)
+ {
+ append_string_object(createFunc, "AS %{definition}L",
+ source);
+ }
+ else
+ {
+ append_string_object(createFunc, "AS %{objfile}L", probin);
+ append_string_object(createFunc, ", %{symbol}L", source);
+ }
+
+ ReleaseSysCache(langTup);
+ ReleaseSysCache(procTup);
+
+ return createFunc;
+}
+
+/*
+ * Deparse a AlterFunctionStmt (ALTER FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the alter command.
+ */
+static ObjTree *
+deparse_AlterFunction(Oid objectId, Node *parsetree)
+{
+ AlterFunctionStmt *node = (AlterFunctionStmt *) parsetree;
+ ObjTree *alterFunc;
+ ObjTree *sign;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ List *params;
+ List *elems = NIL;
+ ListCell *cell;
+ int i;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER FUNCTION %{signature}s %{definition: }s
+ */
+ alterFunc = new_objtree_VA("ALTER FUNCTION", 0);
+
+ params = NIL;
+
+ /*
+ * ALTER FUNCTION does not change signature so we can use catalog
+ * to get input type Oids.
+ */
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *tmp = new_objtree_VA("%{type}T", 0);
+
+ append_object_object(tmp, "type",
+ new_objtree_for_type(procForm->proargtypes.values[i], -1));
+ params = lappend(params, new_object_object(tmp));
+ }
+
+ sign = new_objtree("");
+
+ append_object_object(sign, "%{identity}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+ append_array_object(sign, "(%{arguments:, }s)", params);
+
+ append_object_object(alterFunc, "%{signature}s", sign);
+
+ foreach(cell, node->actions)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "volatility") == 0)
+ {
+ tmp = new_objtree_VA(strVal(defel->arg), 0);
+ }
+ else if (strcmp(defel->defname, "strict") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT", 0);
+ }
+ else if (strcmp(defel->defname, "security") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "SECURITY DEFINER" : "SECURITY INVOKER", 0);
+ }
+ else if (strcmp(defel->defname, "leakproof") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "LEAKPROOF" : "NOT LEAKPROOF", 0);
+ }
+ else if (strcmp(defel->defname, "cost") == 0)
+ {
+ tmp = new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "rows") == 0)
+ {
+ tmp = new_objtree_VA("ROWS", 0);
+ if (defGetNumeric(defel) == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n",
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ }
+ else if (strcmp(defel->defname, "support") == 0)
+ {
+ Oid argtypes[1];
+
+ tmp = new_objtree("SUPPORT");
+
+ Assert(procForm->prosupport);
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else if (strcmp(defel->defname, "parallel") == 0)
+ {
+ char *fmt = psprintf("PARALLEL %s", strVal(defel->arg));
+ tmp = new_objtree(fmt);
+ }
+
+ elems = lappend(elems, new_object_object(tmp));
+ }
+
+ append_array_object(alterFunc, "%{definition: }s", elems);
+
+ ReleaseSysCache(procTup);
+
+ return alterFunc;
+}
+
+/*
+ * Deparse a AlterOwnerStmt (ALTER ... OWNER TO ...).
+ */
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+
+ ownerStmt = new_objtree_VA(fmt, 2,
+ "identity", ObjTypeString,
+ getObjectIdentity(&address, false),
+ "newowner", ObjTypeString,
+ get_rolespec_name(node->newowner));
+
+ return ownerStmt;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Deparse a RenameStmt.
+ */
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+ char *fmtstr;
+ const char *objtype;
+ Relation relation;
+ Oid schemaId;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_TABLE:
+ case OBJECT_INDEX:
+ case OBJECT_SEQUENCE:
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER %s %{if_exists}s %{identity}D RENAME TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+ renameStmt = new_objtree_VA(fmtstr, 0);
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+ append_string_object(renameStmt, "RENAME TO %{newname}I",
+ node->newname);
+ relation_close(relation, AccessShareLock);
+ break;
+
+ case OBJECT_ATTRIBUTE:
+ case OBJECT_COLUMN:
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER objtype %{identity}D RENAME ATTRIBUTE
+ * %{colname}I TO %{newname}I %{cascade}s
+ *
+ * Normal types: ALTER objtype %{if_exists}s %%{identity}D RENAME
+ * COLUMN %{colname}I TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ renameStmt = new_objtree(fmtstr);
+
+ /* Composite types do not support IF EXISTS */
+ if (node->renameType == OBJECT_COLUMN)
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+
+ fmtstr = psprintf("RENAME %s %%{colname}I", objtype);
+
+ append_string_object(renameStmt, fmtstr, node->subname);
+ append_string_object(renameStmt, "TO %{newname}I", node->newname);
+
+ if (node->renameType == OBJECT_ATTRIBUTE)
+ append_object_object(renameStmt, "%{cascade}s",
+ new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool,
+ node->behavior == DROP_CASCADE));
+
+ relation_close(relation, AccessShareLock);
+
+ break;
+ case OBJECT_SCHEMA:
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 1,
+ "identity", ObjTypeString, node->subname,
+ "newname", ObjTypeString, node->newname);
+ break;
+ case OBJECT_TRIGGER:
+ {
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ Relation pg_trigger;
+
+ pg_trigger = relation_open(TriggerRelationId, AccessShareLock);
+ trigTup = get_catalog_object_by_oid(pg_trigger, get_object_attnum_oid(address.classId), address.objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ renameStmt = new_objtree_VA("ALTER TRIGGER %{triggername}I ON %{identity}D RENAME TO %{newname}I",
+ 3,
+ "triggername", ObjTypeString, node->subname,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid),
+ "newname", ObjTypeString, node->newname);
+
+ relation_close(pg_trigger, AccessShareLock);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ return renameStmt;
+}
+
+/*
+ * Deparse the sequence CACHE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cache;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ cache = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(cache);
+}
+
+/*
+ * Deparse the sequence CYCLE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cycle;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ cycle = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(cycle);
+}
+
+/*
+ * Deparse the sequence INCREMENT BY option.
+ */
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *incrementalby;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ incrementalby = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(incrementalby);
+}
+
+/*
+ * Deparse the sequence MINVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *minvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ minvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(minvalue);
+}
+
+/*
+ * Deparse the sequence MAXVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *maxvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ maxvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(maxvalue);
+}
+
+/*
+ * Deparse the sequence START WITH option.
+ */
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *startwith;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ startwith = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(startwith);
+}
+
+/*
+ * Deparse the sequence RESTART option.
+ */
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *restart;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ restart = new_objtree_VA("RESTART %{value}s", 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(restart);
+}
+
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s
+ */
+ createSchema = new_objtree("CREATE SCHEMA");
+
+ append_string_object(createSchema, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(createSchema, "%{name}I", node->schemaname);
+
+ auth = new_objtree("AUTHORIZATION");
+ if (node->authrole)
+ append_string_object(auth, "%{authorization_role}I",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "%{authorization_role}I ");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "%{authorization}s", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/*
+ * Deparse a CreateSeqStmt.
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s SEQUENCE %{identity}D
+ */
+ createSeq = new_objtree("CREATE");
+
+ append_string_object(createSeq, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "SEQUENCE %{identity}D", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * Deparse an AlterSeqStmt.
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq = new_objtree("ALTER SEQUENCE");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "%{identity}D", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+/*
+ * Deparse all the collected subcommands and return an ObjTree representing the
+ * alter command.
+ */
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER reltype %{identity}D %{subcmds:, }s
+ */
+ fmtstr = psprintf("ALTER %s", reltype);
+ alterTableStmt = new_objtree(fmtstr);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "%{identity}D", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+
+ /*
+ * Verbose syntax
+ *
+ * ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX
+ * %index_name}I %{deferrable}s %{init_deferred}s
+ */
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I",
+ 2,
+ "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+
+ append_string_object(tmp, "%{constraint_type}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "USING INDEX %{index_name}I",
+ RelationGetRelationName(idx));
+ append_string_object(tmp, "%{deferrable}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "%{init_deferred}s", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 2,
+ "type", ObjTypeString, "drop default",
+ "column", ObjTypeString, subcmd->name);
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT",
+ 2,
+ "type", ObjTypeString, "set default",
+ "column", ObjTypeString, subcmd->name);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "%{definition}s",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 2,
+ "type", ObjTypeString, "drop not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 2,
+ "type", ObjTypeString, "set not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION",
+ 2,
+ "type", ObjTypeString, "drop expression",
+ "column", ObjTypeString, subcmd->name);
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS": "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3,
+ "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3,
+ "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* If no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 4,
+ "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "deferrable", ObjTypeString,
+ c->deferrable ? "DEFERRABLE" : "NOT DEFERRABLE",
+ "init_deferred", ObjTypeString,
+ c->initdeferred ? "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s ATTRIBUTE %{cascade}s
+ *
+ * Normal types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s COLUMN %{using}s
+ */
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE",
+ istype ? "ATTRIBUTE" : "COLUMN");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* Add the TYPE clause */
+ append_object_object(tmp, "%{datatype}T",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* Add a COLLATE clause, if needed */
+ tmp2 = new_objtree("COLLATE");
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{collation}s", tmp2);
+
+ /* If not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree("USING");
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "%{expression}s",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{using}s", tmp2);
+ }
+
+ /* If it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree("CASCADE");
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I", 2,
+ "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T", 2,
+ "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF", 1,
+ "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "%{ident}s", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "%{ident}s", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "%{ident}s", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "%{ident}s", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D %{partition_bound}s",
+ 3,
+ "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId),
+ "partition_bound", ObjTypeString,
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2,
+ "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2,
+ "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2,
+ "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I",
+ 2,
+ "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "%{definition}s", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY",
+ 2,
+ "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name);
+
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS" : "");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "%{subcmds:, }s", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateTrigStmt:
+ command = deparse_CreateTrigStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ case T_CreateFunctionStmt:
+ command = deparse_CreateFunction(objectId, parsetree);
+ break;
+
+ case T_AlterFunctionStmt:
+ command = deparse_AlterFunction(objectId, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+/*
+ * Handle deparsing of DROP commands.
+ */
+char *
+deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ char *identity = (char *) objidentity;
+ char *fmt;
+ ObjTree *stmt, *stmt2;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+
+ fmt = psprintf("DROP %s IF EXISTS %%{objidentity}s", objecttype);
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString, identity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+
+ append_object_object(stmt, "%{cascade}s", stmt2);
+
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+
+ return command;
+}
+
+/*
+ * Workhorse to deparse a CollectedCommand.
+ */
+char *
+deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ verbose = verbose_mode;
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd, true);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..d64719b
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,760 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo buf, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvBool, an error is raised. If it doesn't
+ * exist, tv_absent is returned; otherwise return the actual trivalue.
+ */
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised unless missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo buf)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * Found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /*
+ * Validate that we got an array if the format string specified one.
+ * And finally print out the data
+ */
+ if (is_array)
+ expand_jsonb_array(buf, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(buf, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and may contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * Expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal.
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* Easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar = dqnextchar % (sizeof(dqsuffixes) - 1);
+ }
+ /* Add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity.
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo buf, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(buf, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(buf, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(buf, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(buf, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(buf, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(buf, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(buf, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(buf, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ bool first = true;
+ StringInfoData arrayelem;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if (!JsonContainerIsArray(container))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ initStringInfo(&arrayelem);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ if (type == WJB_ELEM)
+ {
+ resetStringInfo(&arrayelem);
+
+ if (expand_one_jsonb_element(&arrayelem, param, &v, specifier, NULL))
+ {
+ if (!first)
+ appendStringInfoString(buf, arraysep);
+
+ appendBinaryStringInfo(buf, arrayelem.data, arrayelem.len);
+ first = false;
+ }
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+/*
+ * Workhorse for ddl_deparse_expand_command.
+ */
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo buf = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(buf);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, buf);
+
+ return buf->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable.
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index b0b2118..289a5de 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1700,6 +1700,39 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ /* Open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ef5b34a..39d8d1e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5189,11 +5189,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
- context);
+ address = ATExecAttachPartition(wqueue, rel,
+ (PartitionCmd *) cmd->def,
+ context);
else
- ATExecAttachPartitionIdx(wqueue, rel,
- ((PartitionCmd *) cmd->def)->name);
+ address = ATExecAttachPartitionIdx(wqueue, rel,
+ ((PartitionCmd *) cmd->def)->name);
break;
case AT_DetachPartition:
cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode,
@@ -5201,12 +5202,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
Assert(cmd != NULL);
/* ATPrepCmd ensures it must be a table */
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- ATExecDetachPartition(wqueue, tab, rel,
- ((PartitionCmd *) cmd->def)->name,
- ((PartitionCmd *) cmd->def)->concurrent);
+ address = ATExecDetachPartition(wqueue, tab, rel,
+ ((PartitionCmd *) cmd->def)->name,
+ ((PartitionCmd *) cmd->def)->concurrent);
break;
case AT_DetachPartitionFinalize:
- ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
+ address = ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
diff --git a/src/backend/utils/adt/format_type.c b/src/backend/utils/adt/format_type.c
index 060fd7e..6af2257 100644
--- a/src/backend/utils/adt/format_type.c
+++ b/src/backend/utils/adt/format_type.c
@@ -27,7 +27,6 @@
#include "utils/numeric.h"
#include "utils/syscache.h"
-static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
/*
@@ -367,7 +366,7 @@ format_type_with_typemod(Oid type_oid, int32 typemod)
/*
* Add typmod decoration to the basic type name
*/
-static char *
+char *
printTypmod(const char *typname, int32 typmod, Oid typmodout)
{
char *res;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 26cf4fa..05f78c1 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -357,7 +357,6 @@ static int print_function_arguments(StringInfo buf, HeapTuple proctup,
bool print_table_args, bool print_defaults);
static void print_function_rettype(StringInfo buf, HeapTuple proctup);
static void print_function_trftypes(StringInfo buf, HeapTuple proctup);
-static void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
static void set_rtable_names(deparse_namespace *dpns, List *parent_namespaces,
Bitmapset *rels_used);
static void set_deparse_for_query(deparse_namespace *dpns, Query *query,
@@ -487,22 +486,15 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
static char *generate_relation_name(Oid relid, List *namespaces);
static char *generate_qualified_relation_name(Oid relid);
-static char *generate_function_name(Oid funcid, int nargs,
- List *argnames, Oid *argtypes,
- bool has_variadic, bool *use_variadic_p,
- ParseExprKind special_exprkind);
static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1024,65 +1016,12 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
if (!isnull)
{
Node *qual;
- char relkind;
- deparse_context context;
- deparse_namespace dpns;
- RangeTblEntry *oldrte;
- RangeTblEntry *newrte;
-
- appendStringInfoString(&buf, "WHEN (");
+ char *qualstr;
qual = stringToNode(TextDatumGetCString(value));
+ qualstr = pg_get_trigger_whenclause(trigrec, qual, pretty);
- relkind = get_rel_relkind(trigrec->tgrelid);
-
- /* Build minimal OLD and NEW RTEs for the rel */
- oldrte = makeNode(RangeTblEntry);
- oldrte->rtekind = RTE_RELATION;
- oldrte->relid = trigrec->tgrelid;
- oldrte->relkind = relkind;
- oldrte->rellockmode = AccessShareLock;
- oldrte->alias = makeAlias("old", NIL);
- oldrte->eref = oldrte->alias;
- oldrte->lateral = false;
- oldrte->inh = false;
- oldrte->inFromCl = true;
-
- newrte = makeNode(RangeTblEntry);
- newrte->rtekind = RTE_RELATION;
- newrte->relid = trigrec->tgrelid;
- newrte->relkind = relkind;
- newrte->rellockmode = AccessShareLock;
- newrte->alias = makeAlias("new", NIL);
- newrte->eref = newrte->alias;
- newrte->lateral = false;
- newrte->inh = false;
- newrte->inFromCl = true;
-
- /* Build two-element rtable */
- memset(&dpns, 0, sizeof(dpns));
- dpns.rtable = list_make2(oldrte, newrte);
- dpns.subplans = NIL;
- dpns.ctes = NIL;
- dpns.appendrels = NULL;
- set_rtable_names(&dpns, NIL, NULL);
- set_simple_column_names(&dpns);
-
- /* Set up context with one-deep namespace stack */
- context.buf = &buf;
- context.namespaces = list_make1(&dpns);
- context.windowClause = NIL;
- context.windowTList = NIL;
- context.varprefix = true;
- context.prettyFlags = GET_PRETTY_FLAGS(pretty);
- context.wrapColumn = WRAP_COLUMN_DEFAULT;
- context.indentLevel = PRETTYINDENT_STD;
- context.special_exprkind = EXPR_KIND_NONE;
- context.appendparents = NULL;
-
- get_rule_expr(qual, &context, false);
-
- appendStringInfoString(&buf, ") ");
+ appendStringInfo(&buf, "WHEN (%s) ", qualstr);
}
appendStringInfo(&buf, "EXECUTE FUNCTION %s(",
@@ -1123,6 +1062,64 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
return buf.data;
}
+char *
+pg_get_trigger_whenclause(Form_pg_trigger trigrec, Node *whenClause, bool pretty)
+{
+ StringInfoData buf;
+ char relkind;
+ deparse_context context;
+ deparse_namespace dpns;
+ RangeTblEntry *oldrte;
+ RangeTblEntry *newrte;
+
+ initStringInfo(&buf);
+
+ relkind = get_rel_relkind(trigrec->tgrelid);
+
+ /* Build minimal OLD and NEW RTEs for the rel */
+ oldrte = makeNode(RangeTblEntry);
+ oldrte->rtekind = RTE_RELATION;
+ oldrte->relid = trigrec->tgrelid;
+ oldrte->relkind = relkind;
+ oldrte->alias = makeAlias("old", NIL);
+ oldrte->eref = oldrte->alias;
+ oldrte->lateral = false;
+ oldrte->inh = false;
+ oldrte->inFromCl = true;
+
+ newrte = makeNode(RangeTblEntry);
+ newrte->rtekind = RTE_RELATION;
+ newrte->relid = trigrec->tgrelid;
+ newrte->relkind = relkind;
+ newrte->alias = makeAlias("new", NIL);
+ newrte->eref = newrte->alias;
+ newrte->lateral = false;
+ newrte->inh = false;
+ newrte->inFromCl = true;
+
+ /* Build two-element rtable */
+ memset(&dpns, 0, sizeof(dpns));
+ dpns.rtable = list_make2(oldrte, newrte);
+ dpns.ctes = NIL;
+ set_rtable_names(&dpns, NIL, NULL);
+ set_simple_column_names(&dpns);
+
+ /* Set up context with one-deep namespace stack */
+ context.buf = &buf;
+ context.namespaces = list_make1(&dpns);
+ context.windowClause = NIL;
+ context.windowTList = NIL;
+ context.varprefix = true;
+ context.prettyFlags = pretty ? PRETTYFLAG_PAREN | PRETTYFLAG_INDENT : PRETTYFLAG_INDENT;
+ context.wrapColumn = WRAP_COLUMN_DEFAULT;
+ context.indentLevel = PRETTYINDENT_STD;
+ context.special_exprkind = EXPR_KIND_NONE;
+
+ get_rule_expr(whenClause, &context, false);
+
+ return buf.data;
+}
+
/* ----------
* pg_get_indexdef - Get the definition of an index
*
@@ -1890,6 +1887,13 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+/* Internal version that reports the full partition key definition */
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2143,6 +2147,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -3512,7 +3525,7 @@ pg_get_function_arg_default(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(string_to_text(str));
}
-static void
+void
print_function_sqlbody(StringInfo buf, HeapTuple proctup)
{
int numargs;
@@ -12017,7 +12030,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12411,7 +12424,7 @@ generate_qualified_relation_name(Oid relid)
*
* The result includes all necessary quoting and schema-prefixing.
*/
-static char *
+char *
generate_function_name(Oid funcid, int nargs, List *argnames, Oid *argtypes,
bool has_variadic, bool *use_variadic_p,
ParseExprKind special_exprkind)
@@ -12797,7 +12810,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e41f4d..e38230d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11886,4 +11886,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'deparse the DDL command into json format string',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index d38c0e2..4ca84b2 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..bdfdf47
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.h
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/ddl_deparse.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd, bool verbose_mode);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior);
+
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 221c3e6..85a62a1 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid, int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
extern int32 type_maximum_size(Oid type_oid, int32 typemod);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..3d69a60 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -13,6 +13,7 @@
#ifndef RULEUTILS_H
#define RULEUTILS_H
+#include "catalog/pg_trigger.h"
#include "nodes/nodes.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
@@ -23,12 +24,16 @@ struct PlannedStmt;
extern char *pg_get_indexdef_string(Oid indexrelid);
extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
+extern char *pg_get_trigger_whenclause(Form_pg_trigger trigrec,
+ Node *whenClause, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -40,8 +45,16 @@ extern List *select_rtable_names_for_explain(List *rtable,
Bitmapset *rels_used);
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
+extern char *generate_function_name(Oid funcid, int nargs, List *argnames,
+ Oid *argtypes, bool has_variadic,
+ bool *use_variadic_p,
+ ParseExprKind special_exprkind);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
+extern void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
#endif /* RULEUTILS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 34a76ce..8ed4f8c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1621,6 +1621,9 @@ OSInfo
OSSLCipher
OSSLDigest
OVERLAPPED
+ObjElem
+ObjTree
+ObjType
ObjectAccessDrop
ObjectAccessNamespaceSearch
ObjectAccessPostAlter
@@ -3182,6 +3185,7 @@ colormaprange
compare_context
config_var_value
contain_aggs_of_level_context
+convSpecifier
convert_testexpr_context
copy_data_source_cb
core_YYSTYPE
--
2.7.2.windows.1
[application/octet-stream] v13-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB5716FCCEF8009E4464042476948E9@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v13-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From b25920a8213e38de9eb8bb096bdc29d64f3e9482 Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
1.8.3.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-22 21:18 Zheng Li <zhengli10@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 2 replies; 449+ messages in thread
From: Zheng Li @ 2022-07-22 21:18 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
Hello,
Here is a patch that supports replication of global object commands,
these include ROLE statements, database statements and tablespace statements.
The patch should be applied on top of the v13 DDL replication patch set that
ZJ Hou sent in the previous email.
Global objects commands are different from other DDL commands in
that:
1. Global objects commands are allowed to be executed in any databases
2. Global objects are not schema qualified
2. Global objects commands are not captured by event triggers
This patch supports global objects commands replication by WAL
logging the command using the same function for DDL logging -
LogLogicalDDLMessage, towards the end of standard_ProcessUtility.
Because global objects are not schema qualified, we can skip the deparser
invocation and directly log the original command string for replay on
the subscriber.
A key problem is global objects can get inconsistent between the
publisher and the subscriber if a command changes the global object
in a database (on the source side) which doesn't configure logical replication.
I think we can work on the following directions in order to avoid such
inconsistency:
1. Introduce a publication option for global objects command replication
and document that logical replication of global objects commands is preferred
to be configured on all databases. Otherwise inconsistency can happen
if a command changes the global object in a database which doesn't configure
logical replication.
2. Introduce database cluster level logical replication to avoid such
inconsistency,
this is especially handy when there is a large number of databases to
configure for logical
replication.
Regards,
Zheng
Attachments:
[application/octet-stream] 0005-Support-replication-of-global-object-commands-these-.patch (8.9K, ../../CAAD30ULBUSHG4n-_zZ7yLGPri-LkS0o+vp64WCKj-mFcF27ekg@mail.gmail.com/2-0005-Support-replication-of-global-object-commands-these-.patch)
download | inline diff:
From 863ac6e33780b8ab982fa658ab57e5418cc39b79 Mon Sep 17 00:00:00 2001
From: "Zheng (Zane) Li" <zhelli@amazon.com>
Date: Fri, 22 Jul 2022 18:34:32 +0000
Subject: [PATCH 5/5] Support replication of global object commands, these
include ROLE statements, database statements and tablespace statements.
Global objects commands are different from other DDL commands in
that:
1. Global objects commands are allowed to be executed in any databases
2. Global objects are not schema qualified
2. Global objects commands are not captured by event triggers
This patch supports global objects commands replication by WAL
logging the command using the same function for DDL logging, i.e.
LogLogicalDDLMessage. Because global objects are not schema qualified,
we skip the deparser invocation and directly log the original
command string for replay on the subscriber.
A key problem is global objects can get inconsistent between the
publisher and the subscriber if a command changes the global object
in a database which doesn't configure logical replication. I think
we can work on the following in order to avoid such inconsistency:
1. Introduce a publication option for global objects command replication
and document that logical replication of global objects command need
to be configured on all databases, otherwise inconsistency can happen
if a command changes the global object in a database which doesn't configure
logical replication.
2. Introduce database cluster level logical replication, this is handy
when there is a large number of databases to configure for logical
replication.
---
src/backend/replication/logical/worker.c | 6 +-
src/backend/replication/pgoutput/pgoutput.c | 3 +
src/backend/tcop/utility.c | 44 +++++++++++++
.../subscription/t/032_ddl_replication.pl | 64 ++++++++++++++-----
4 files changed, 99 insertions(+), 18 deletions(-)
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 443a3c2e0a..2bcc8df9a1 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -2587,7 +2587,11 @@ apply_handle_ddl(StringInfo s)
/* Make sure we are in a transaction command */
begin_replication_step();
- ddl_command = ddl_deparse_json_to_string(message);
+ if (strcmp(prefix, "deparse") == 0)
+ ddl_command = ddl_deparse_json_to_string(message);
+ else if (strcmp(prefix, "cmd") == 0)
+ ddl_command = message;
+
debug_query_string = ddl_command;
/* DestNone for logical replication */
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index 7bf63dc4d4..a19b77c7ee 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -1868,6 +1868,9 @@ pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
case DCT_ObjectDrop:
/* do nothing */
break;
+ case DCT_GlobalObjectCmd:
+ /* do nothing */
+ break;
default:
elog(ERROR, "unsupported type %d", cmdtype);
break;
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 490b73b66e..f816424add 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -64,6 +64,7 @@
#include "postmaster/bgwriter.h"
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteRemove.h"
+#include "replication/ddlmessage.h"
#include "storage/fd.h"
#include "tcop/pquery.h"
#include "tcop/utility.h"
@@ -532,6 +533,48 @@ ProcessUtility(PlannedStmt *pstmt,
dest, qc);
}
+/*
+ * Log a global object DDL command for logical replication
+ */
+
+static void
+LogGlobalObjectCommand(Node *parsetree, const char * queryString)
+{
+ switch (nodeTag(parsetree))
+ {
+ /* ROLE statements */
+ case T_CreateRoleStmt:
+ case T_AlterRoleStmt:
+ case T_AlterRoleSetStmt:
+ case T_DropRoleStmt:
+ case T_ReassignOwnedStmt:
+ case T_GrantRoleStmt:
+
+ /* Database statements */
+ case T_CreatedbStmt:
+ case T_AlterDatabaseStmt:
+ case T_AlterDatabaseRefreshCollStmt:
+ case T_AlterDatabaseSetStmt:
+ case T_DropdbStmt:
+
+ /* TableSpace statements */
+ case T_CreateTableSpaceStmt:
+ case T_DropTableSpaceStmt:
+ case T_AlterTableSpaceOptionsStmt:
+ if (XLogLogicalInfoActive())
+ {
+ LogLogicalDDLMessage("cmd",
+ InvalidOid,
+ DCT_GlobalObjectCmd,
+ queryString,
+ strlen(queryString) + 1);
+ }
+ break;
+ default:
+ break;
+ }
+}
+
/*
* standard_ProcessUtility itself deals only with utility commands for
* which we do not provide event trigger support. Commands that do have
@@ -1077,6 +1120,7 @@ standard_ProcessUtility(PlannedStmt *pstmt,
break;
}
+ LogGlobalObjectCommand(parsetree, queryString);
free_parsestate(pstate);
/*
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
index 94de8edd08..d38b8d53e5 100644
--- a/src/test/subscription/t/032_ddl_replication.pl
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -378,23 +378,23 @@ $node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
# Prepare the directories for the publisher and subscriber first.
-my ($basedir, $tablespace_dir);
-
-$basedir = $node_publisher->basedir;
-$tablespace_dir = "$basedir/tblspc_pub";
-mkdir($tablespace_dir);
-$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
-$basedir = $node_subscriber->basedir;
-$tablespace_dir = "$basedir/tblspc_sub";
-mkdir ($tablespace_dir);
-$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
-
-$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
-$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
-$node_publisher->wait_for_catchup('mysub');
-$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
-is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
-$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+#my ($basedir, $tablespace_dir);
+
+#$basedir = $node_publisher->basedir;
+#$tablespace_dir = "$basedir/tblspc_pub";
+#mkdir($tablespace_dir);
+#$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+#$basedir = $node_subscriber->basedir;
+#$tablespace_dir = "$basedir/tblspc_sub";
+#mkdir ($tablespace_dir);
+#$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+#$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+#$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+#$node_publisher->wait_for_catchup('mysub');
+#$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+#is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+#$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
# Test CREATE TABLE OF (creating a type is not replicated)
$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
@@ -406,6 +406,36 @@ $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
is($result, qq(1), 'CREATE TABLE OF replicated');
$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+# Test CREATE ROLE is replicated
+$node_publisher->safe_psql('postgres', "CREATE ROLE test_user REPLICATION LOGIN;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT COUNT(*) from pg_shadow where usename = 'test_user' and userepl = 't'");
+is($result, qq(1), 'CREATE ROLE replicated');
+
+# Test ALTER ROLE is replicated
+$node_publisher->safe_psql('postgres', "ALTER ROLE test_user NOREPLICATION LOGIN;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT COUNT(*) from pg_shadow where usename = 'test_user' and userepl = 'f'");
+is($result, qq(1), 'ALTER ROLE replicated');
+
+# Test CREATE DATABASE is replicated
+$node_publisher->safe_psql('postgres', "CREATE DATABASE db1;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_database WHERE datname = 'db1'");
+is($result, qq(1), 'CREATE DATABASE replicated');
+
+# Test ALTER DATABASE is replicated
+$node_publisher->safe_psql('postgres', "ALTER DATABASE db1 CONNECTION LIMIT 10;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT datconnlimit FROM pg_database WHERE datname = 'db1'");
+is($result, qq(10), 'ALTER DATABASE is replicated');
+
+# Test DROP DATABASE is replicated
+$node_publisher->safe_psql('postgres', "DROP DATABASE db1;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM pg_database WHERE datname = 'db1'");
+is($result, qq(0), 'DROP DATABASE is replicated');
+
pass "DDL replication tests passed:";
$node_subscriber->stop;
--
2.32.0
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-23 15:33 Joe Conway <mail@joeconway.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Joe Conway @ 2022-07-23 15:33 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
On 7/22/22 17:18, Zheng Li wrote:
> Here is a patch that supports replication of global object commands,
> these include ROLE statements, database statements and tablespace statements.
> The patch should be applied on top of the v13 DDL replication patch set that
> ZJ Hou sent in the previous email.
>
> Global objects commands are different from other DDL commands in
> that:
> 1. Global objects commands are allowed to be executed in any databases
> 2. Global objects are not schema qualified
> 2. Global objects commands are not captured by event triggers
>
> This patch supports global objects commands replication by WAL
> logging the command using the same function for DDL logging -
> LogLogicalDDLMessage, towards the end of standard_ProcessUtility.
> Because global objects are not schema qualified, we can skip the deparser
> invocation and directly log the original command string for replay on
> the subscriber.
I have not looked at the patch but +1 for the general concept. Seems
like you might want to start a separate thread, perhaps after the
currently running commitfest is over.
> A key problem is global objects can get inconsistent between the
> publisher and the subscriber if a command changes the global object
> in a database (on the source side) which doesn't configure logical replication.
> I think we can work on the following directions in order to avoid such
> inconsistency:
>
> 1. Introduce a publication option for global objects command replication
> and document that logical replication of global objects commands is preferred
> to be configured on all databases. Otherwise inconsistency can happen
> if a command changes the global object in a database which doesn't configure
> logical replication.
>
> 2. Introduce database cluster level logical replication to avoid
> such inconsistency, this is especially handy when there is a large
> number of databases to configure for logical replication.
I would strongly favor #2, although I admittedly have no idea what
complexities it adds.
--
Joe Conway
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-23 16:23 Zheng Li <zhengli10@gmail.com>
parent: Joe Conway <mail@joeconway.com>
0 siblings, 0 replies; 449+ messages in thread
From: Zheng Li @ 2022-07-23 16:23 UTC (permalink / raw)
To: Joe Conway <mail@joeconway.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
On Sat, Jul 23, 2022 at 11:33 AM Joe Conway <mail@joeconway.com> wrote:
>
> On 7/22/22 17:18, Zheng Li wrote:
> > Here is a patch that supports replication of global object commands,
> > these include ROLE statements, database statements and tablespace statements.
> > The patch should be applied on top of the v13 DDL replication patch set that
> > ZJ Hou sent in the previous email.
> >
> > Global objects commands are different from other DDL commands in
> > that:
> > 1. Global objects commands are allowed to be executed in any databases
> > 2. Global objects are not schema qualified
> > 2. Global objects commands are not captured by event triggers
> >
> > This patch supports global objects commands replication by WAL
> > logging the command using the same function for DDL logging -
> > LogLogicalDDLMessage, towards the end of standard_ProcessUtility.
> > Because global objects are not schema qualified, we can skip the deparser
> > invocation and directly log the original command string for replay on
> > the subscriber.
>
> I have not looked at the patch but +1 for the general concept. Seems
> like you might want to start a separate thread, perhaps after the
> currently running commitfest is over.
Thanks for the suggestion. I'll start a new thread on the replication
of global objects
commands. I think it's different enough to get its own attention.
>
> > A key problem is global objects can get inconsistent between the
> > publisher and the subscriber if a command changes the global object
> > in a database (on the source side) which doesn't configure logical replication.
> > I think we can work on the following directions in order to avoid such
> > inconsistency:
> >
> > 1. Introduce a publication option for global objects command replication
> > and document that logical replication of global objects commands is preferred
> > to be configured on all databases. Otherwise inconsistency can happen
> > if a command changes the global object in a database which doesn't configure
> > logical replication.
> >
> > 2. Introduce database cluster level logical replication to avoid
> > such inconsistency, this is especially handy when there is a large
> > number of databases to configure for logical replication.
>
> I would strongly favor #2, although I admittedly have no idea what
> complexities it adds.
I will also start a new thread once we have more concrete plans on this.
Regards,
Zheng
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-07-25 06:13 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Amit Kapila @ 2022-07-25 06:13 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
On Sat, Jul 23, 2022 at 2:49 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> Hello,
>
> Here is a patch that supports replication of global object commands,
> these include ROLE statements, database statements and tablespace statements.
> The patch should be applied on top of the v13 DDL replication patch set that
> ZJ Hou sent in the previous email.
>
> Global objects commands are different from other DDL commands in
> that:
> 1. Global objects commands are allowed to be executed in any databases
> 2. Global objects are not schema qualified
> 2. Global objects commands are not captured by event triggers
>
> This patch supports global objects commands replication by WAL
> logging the command using the same function for DDL logging -
> LogLogicalDDLMessage, towards the end of standard_ProcessUtility.
>
I noticed that LogLogicalDDLMessage() uses MyDatabaseId and then
decoding can take some action (filtering) based on that. Is it Okay to
use that function for global objects, if so, you might want to add a
comment for the same?
> Because global objects are not schema qualified, we can skip the deparser
> invocation and directly log the original command string for replay on
> the subscriber.
>
> A key problem is global objects can get inconsistent between the
> publisher and the subscriber if a command changes the global object
> in a database (on the source side) which doesn't configure logical replication.
> I think we can work on the following directions in order to avoid such
> inconsistency:
>
> 1. Introduce a publication option for global objects command replication
> and document that logical replication of global objects commands is preferred
> to be configured on all databases. Otherwise inconsistency can happen
> if a command changes the global object in a database which doesn't configure
> logical replication.
>
> 2. Introduce database cluster level logical replication to avoid such
> inconsistency,
> this is especially handy when there is a large number of databases to
> configure for logical
> replication.
>
In general, I agree with your comment below that we can work on this
after we have some more concrete plans/discussions. I think we can
probably consider this when we have more discussion around the
publication commands for the DDL objects. However, it would be good if
you can add some more details about the above two options.
As per my understanding, the overall work on this project includes the
following sub-tasks:
a. DDL Deparsing: This is required to support DDL replication of
non-global objects. The work for this is in progress, this is based on
prior work by Alvaro.
b. DDL Replication: This includes replication of DDL commands based on
event triggers and DDL deparsing. The work on this is also in
progress.
c. DDL Replication of global objects: It requires a different approach
due to the reasons quoted above in your email. Zheng has started
working on it.
d. Initial Sync: I think there is a brief discussion about this in the
thread but no concrete proposal yet. I think it is important to solve
this part of the puzzle as well to have an overall design ready for
this project. Do let me know if you, Sawada-San, or anybody else
intends to work on it? I think that will avoid overlap of work.
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-07-28 03:48 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 2 replies; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-07-28 03:48 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>; Peter Smith <smithpb2250@gmail.com>
On Wednesday, July 20, 2022 8:28 PM houzj.fnst@fujitsu.com wrote:
> On Tuesday, July 12, 2022 5:04 PM houzj.fnst@fujitsu.com
> <houzj.fnst@fujitsu.com> wrote:
> > On Friday, July 8, 2022 10:26 AM Peter Smith <smithpb2250@gmail.com>
> > wrote:
> >
> >
> > > Here are some review comments for the patch v11-0001:
> >
> > Attach the V12 patch set which include the following changes:
> >
> > * Address comments from peter[1]
> > * Refactor the deparser and provide an option to control whether
> > output the "not present" syntax part.
> > And for DDL replication, we don't WAL log the "not present" syntax
> > string for now.
> > * Address most comments from Vignesh[2] except the one about pg_dump
> > vs the event trigger for ddl replication which need some more research.
>
> Attach the new version patch set which added support for CREATE/ALTER
> FUNCTION, CREATE/ALTER PROCEDURE, CREATE/ALTER TRIGGER, RENAME
> TABLE/INDEX/SEQUENCE.
>
> Thanks to Vignesh for helping off list.
Attach the new version patch set which added support for
CREATE/ALTER OPERATOR [CLASS/FAMILY].
The DROP case need some more handling and I will update it later.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v14-0001-Functions-to-deparse-DDL-commands.patch (184.3K, ../../OS0PR01MB57168B82B65B491596F5832894969@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v14-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 37a4c6d2317f04bc9bd9dbf0ede87e0cabcfce84 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Fri, 27 May 2022 13:47:05 +0800
Subject: [PATCH] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides a base for logical replication of DDL statements. Currently,
the patch has support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/ALTER/DROP SCHEMA
CREATE/ALTER FUNCTION/PROCEDURE
CREATE/ALTER TRIGGER
CREATE/DROP INDEX
CREATE/DROP OPERATOR [CLASS/FAMILY]
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 5134 +++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 760 ++++++
src/backend/commands/sequence.c | 33 +
src/backend/commands/tablecmds.c | 17 +-
src/backend/utils/adt/format_type.c | 3 +-
src/backend/utils/adt/ruleutils.c | 149 +-
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 23 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/builtins.h | 1 +
src/include/utils/ruleutils.h | 13 +
src/tools/pgindent/typedefs.list | 4 +
14 files changed, 6069 insertions(+), 78 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..f98a544
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,5134 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_language.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_opclass.h"
+#include "catalog/pg_operator.h"
+#include "catalog/pg_opfamily.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "funcapi.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/guc.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+/* Estimated length of the generated jsonb string */
+#define JSONB_ESTIMATED_LEN 128
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+ StringInfo fmtinfo;
+ bool present;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+bool verbose = true;
+
+static ObjElem *new_object(ObjType type, char *name);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static void append_float_object(ObjTree *tree, char *name, float8 value);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+static List *FunctionGetDefaults(text *proargdefaults);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_extended, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
+
+ /*
+ * Special-case crock for types with strange typmod rules where we put
+ * typmod at the middle of name(e.g. TIME(6) with time zone ). We cannot
+ * schema-qualify nor add quotes to the type name in these cases.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmod. */
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
+ break;
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+ }
+ else
+ {
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+ }
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Return the defaults values of arguments to a function, as a list of
+ * deparsed expressions.
+ */
+static List *
+FunctionGetDefaults(text *proargdefaults)
+{
+ List *nodedefs;
+ List *strdefs = NIL;
+ ListCell *cell;
+
+ nodedefs = (List *) stringToNode(TextDatumGetCString(proargdefaults));
+ if (!IsA(nodedefs, List))
+ elog(ERROR, "proargdefaults is not a list");
+
+ foreach(cell, nodedefs)
+ {
+ Node *onedef = lfirst(cell);
+
+ strdefs = lappend(strdefs, deparse_expression(onedef, NIL, false, false));
+ }
+
+ return strdefs;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(char *fmt)
+{
+ ObjTree *params;
+
+ params = palloc0(sizeof(ObjTree));
+ params->present = true;
+ slist_init(¶ms->params);
+
+ if (fmt)
+ {
+ params->fmtinfo = makeStringInfo();
+ appendStringInfoString(params->fmtinfo, fmt);
+ }
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree(fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+ elem = new_object(type, NULL);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ break;
+ case ObjTypeBool:
+ elem->value.boolean = va_arg(args, int);
+ break;
+ case ObjTypeString:
+ elem->value.string = va_arg(args, char *);
+ break;
+ case ObjTypeArray:
+ elem->value.array = va_arg(args, List *);
+ break;
+ case ObjTypeInteger:
+ elem->value.integer = va_arg(args, int64);
+ break;
+ case ObjTypeFloat:
+ elem->value.flt = va_arg(args, double);
+ break;
+ case ObjTypeObject:
+ elem->value.object = va_arg(args, ObjTree *);
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/*
+ * Append the input format string to the ObjTree.
+ */
+static void
+append_format_string(ObjTree *tree, char *sub_fmt)
+{
+ int len;
+ char *fmt;
+
+ if (!tree->fmtinfo)
+ return;
+
+ fmt = tree->fmtinfo->data;
+ len = tree->fmtinfo->len;
+
+ /* Add a separator if necessary */
+ if (len > 0 && fmt[len - 1] != ' ')
+ appendStringInfoSpaces(tree->fmtinfo, 1);
+
+ appendStringInfoString(tree->fmtinfo, sub_fmt);
+}
+
+/*
+ * Return the object name which is extracted from the input "*%{name[:.]}*"
+ * style string. And append the input format string to the ObjTree.
+ */
+static char *
+append_object_to_format_string(ObjTree *tree, char *sub_fmt)
+{
+ StringInfoData object_name;
+ const char *end_ptr;
+ const char *cp;
+ bool start_copy = false;
+
+ if (sub_fmt == NULL || tree->fmtinfo == NULL)
+ return sub_fmt;
+
+ initStringInfo(&object_name);
+ end_ptr = sub_fmt + strlen(sub_fmt);
+
+ for (cp = sub_fmt; cp < end_ptr; cp++)
+ {
+ if (*cp == '{')
+ {
+ start_copy = true;
+ continue;
+ }
+
+ if (!start_copy)
+ continue;
+
+ if (*cp == ':' || *cp == '}')
+ break;
+
+ appendStringInfoCharMacro(&object_name, *cp);
+ }
+
+ if (object_name.len == 0)
+ elog(ERROR, "object name not found");
+
+ append_format_string(tree, sub_fmt);
+
+ return object_name.data;
+}
+
+/* Allocate a new parameter */
+static ObjElem *
+new_object(ObjType type, char *name)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = name;
+ param->objtype = type;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *sub_fmt)
+{
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ append_premade_object(tree, new_object(ObjTypeNull, object_name));
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *sub_fmt, bool value)
+{
+ ObjElem *param;
+ char *object_name = sub_fmt;
+ bool is_present_flag = false;
+
+ Assert(sub_fmt);
+
+ if (strcmp(sub_fmt, "present") == 0)
+ {
+ is_present_flag = true;
+ tree->present = value;
+ }
+
+ if (!verbose && !tree->present)
+ return;
+
+ if (!is_present_flag)
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeBool, object_name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = new_object(ObjTypeString, NULL);
+ param->value.string = value;
+
+ return param;
+}
+
+/* Append a string parameter to a tree */
+static void
+append_string_object(ObjTree *tree, char *sub_fmt, char *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && (value == NULL || value[0] == '\0'))
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeString, object_name);
+ param->value.string = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeObject, NULL);
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *sub_fmt, ObjTree *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && !value->present)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeObject, object_name);
+ param->value.object = value;
+ append_premade_object(tree, param);
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *sub_fmt, List *array)
+{
+ ObjElem *param;
+ List *narray = NIL;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ {
+ ListCell *lc;
+
+ /* Extract the ObjElems whose present flag is true */
+ foreach(lc, array)
+ {
+ ObjElem *elem = (ObjElem *) lfirst(lc);
+
+ Assert(elem->objtype == ObjTypeObject);
+
+ if (elem->value.object->present)
+ narray = lappend(narray, elem);
+ }
+
+ if (list_length(narray) == 0)
+ return;
+ }
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeArray, object_name);
+ param->value.array = narray;
+ append_premade_object(tree, param);
+}
+
+/*
+ * Append a float8 parameter to a tree.
+ */
+static void
+append_float_object(ObjTree *tree, char *sub_fmt, float8 value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeFloat, object_name);
+ param->value.flt = value;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Process the pre-built format string from the ObjTree into the output parse
+ * state.
+ */
+static void
+objtree_fmt_to_jsonb_element(JsonbParseState *state, ObjTree *tree)
+{
+ JsonbValue key;
+ JsonbValue val;
+
+ if (!tree->fmtinfo)
+ return;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.val = "fmt";
+ key.val.string.len = strlen(key.val.string.val);
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the pre-built format string */
+ val.type = jbvString;
+ val.val.string.len = tree->fmtinfo->len;
+ val.val.string.val = tree->fmtinfo->data;
+ pushJsonbValue(&state, WJB_VALUE, &val);
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb.
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ objtree_fmt_to_jsonb_element(state, tree);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (OidIsValid(typnspid))
+ typnsp = get_namespace_name_or_temp(typnspid);
+ else
+ typnsp = pstrdup("");
+
+ typeParam = new_objtree_VA(NULL, 4,
+ "schemaname", ObjTypeString, typnsp,
+ "typename", ObjTypeString, typename,
+ "typmod", ObjTypeString, typmodstr,
+ "typarray", ObjTypeBool, typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements.
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+
+ qualified = new_objtree_VA(NULL, 2,
+ "schemaname", ObjTypeString, namespace,
+ "objname", ObjTypeString, pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId.
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value.
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * Deparse the definition of column identity.
+ */
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *column;
+ ObjTree *identobj;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+ char *identfmt;
+ char *objfmt;
+
+ column = new_objtree("");
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(column, "present", false);
+ return column;
+ }
+
+ if (alter_table)
+ {
+ identfmt = "SET GENERATED ";
+ objfmt = "%{option}s";
+ }
+ else
+ {
+ identfmt = "GENERATED ";
+ objfmt = "%{option}s AS IDENTITY";
+ }
+
+ identobj = new_objtree(identfmt);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(identobj, objfmt, "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(identobj, objfmt, "BY DEFAULT");
+ else
+ append_bool_object(identobj, "present", false);
+
+ append_object_object(column, "%{identity_type}s", identobj);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* Definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* We purposefully do not emit OWNED BY here */
+
+ if (alter_table)
+ append_array_object(column, "%{seq_definition: }s", elems);
+ else
+ append_array_object(column, "( %{seq_definition: }s )", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return column;
+}
+
+/*
+ * Deparse a CreateTrigStmt (CREATE TRIGGER)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateTrigStmt(Oid objectId, Node *parsetree)
+{
+ CreateTrigStmt *node = (CreateTrigStmt *) parsetree;
+ Relation pg_trigger;
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ ObjTree *trigger;
+ ObjTree *tmp;
+ int tgnargs;
+ List *list;
+ List *events;
+
+ pg_trigger = table_open(TriggerRelationId, AccessShareLock);
+
+ trigTup = get_catalog_object_by_oid(pg_trigger, Anum_pg_trigger_oid, objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{constraint}s TRIGGER %{name}I %{time}s %{events: OR }s
+ * ON %{relation}D %{from_table}s %{constraint_attrs: }s
+ * FOR EACH %{for_each}s %{when}s EXECUTE PROCEDURE %{function}s
+ */
+ trigger = new_objtree("CREATE");
+
+ append_string_object(trigger, "%{constraint}s",
+ node->isconstraint ? "CONSTRAINT" : "");
+
+ append_string_object(trigger, "TRIGGER %{name}I", node->trigname);
+
+ if (node->timing == TRIGGER_TYPE_BEFORE)
+ append_string_object(trigger, "%{time}s", "BEFORE");
+ else if (node->timing == TRIGGER_TYPE_AFTER)
+ append_string_object(trigger, "%{time}s", "AFTER");
+ else if (node->timing == TRIGGER_TYPE_INSTEAD)
+ append_string_object(trigger, "%{time}s", "INSTEAD OF");
+ else
+ elog(ERROR, "unrecognized trigger timing value %d", node->timing);
+
+ /*
+ * Decode the events that the trigger fires for. The output is a list;
+ * in most cases it will just be a string with the even name, but when
+ * there's an UPDATE with a list of columns, we return a JSON object.
+ */
+ events = NIL;
+ if (node->events & TRIGGER_TYPE_INSERT)
+ events = lappend(events, new_string_object("INSERT"));
+ if (node->events & TRIGGER_TYPE_DELETE)
+ events = lappend(events, new_string_object("DELETE"));
+ if (node->events & TRIGGER_TYPE_TRUNCATE)
+ events = lappend(events, new_string_object("TRUNCATE"));
+ if (node->events & TRIGGER_TYPE_UPDATE)
+ {
+ if (node->columns == NIL)
+ {
+ events = lappend(events, new_string_object("UPDATE"));
+ }
+ else
+ {
+ ObjTree *update;
+ ListCell *cell;
+ List *cols = NIL;
+
+ /*
+ * Currently only UPDATE OF can be objects in the output JSON, but
+ * we add a "kind" element so that user code can distinguish
+ * possible future new event types.
+ */
+ update = new_objtree_VA("UPDATE OF", 1,
+ "kind", ObjTypeString, "update_of");
+
+ foreach(cell, node->columns)
+ {
+ char *colname = strVal(lfirst(cell));
+
+ cols = lappend(cols,
+ new_string_object(colname));
+ }
+
+ append_array_object(update, "%{columns:, }I", cols);
+
+ events = lappend(events,
+ new_object_object(update));
+ }
+ }
+ append_array_object(trigger, "%{events: OR }s", events);
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid);
+ append_object_object(trigger, "ON %{relation}D", tmp);
+
+ tmp = new_objtree_VA("FROM", 0);
+ if (trigForm->tgconstrrelid)
+ {
+ ObjTree *rel;
+
+ rel = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgconstrrelid);
+ append_object_object(tmp, "%{relation}D", rel);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{from_table}s", tmp);
+
+ list = NIL;
+ if (node->deferrable)
+ list = lappend(list,
+ new_string_object("DEFERRABLE"));
+ if (node->initdeferred)
+ list = lappend(list,
+ new_string_object("INITIALLY DEFERRED"));
+ append_array_object(trigger, "%{constraint_attrs: }s", list);
+
+ append_string_object(trigger, "FOR EACH %{for_each}s",
+ node->row ? "ROW" : "STATEMENT");
+
+ tmp = new_objtree_VA("WHEN", 0);
+ if (node->whenClause)
+ {
+ Node *whenClause;
+ Datum value;
+ bool isnull;
+
+ value = fastgetattr(trigTup, Anum_pg_trigger_tgqual,
+ RelationGetDescr(pg_trigger), &isnull);
+ if (isnull)
+ elog(ERROR, "bogus NULL tgqual");
+
+ whenClause = stringToNode(TextDatumGetCString(value));
+ append_string_object(tmp, "(%{clause}s)",
+ pg_get_trigger_whenclause(trigForm,
+ whenClause,
+ false));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{when}s", tmp);
+
+ tmp = new_objtree_VA("%{funcname}D", 1,
+ "funcname", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ trigForm->tgfoid));
+ list = NIL;
+ tgnargs = trigForm->tgnargs;
+ if (tgnargs > 0)
+ {
+ bytea *tgargs;
+ char *argstr;
+ bool isnull;
+ int findx;
+ int lentgargs;
+ char *p;
+
+ tgargs = DatumGetByteaP(fastgetattr(trigTup,
+ Anum_pg_trigger_tgargs,
+ RelationGetDescr(pg_trigger),
+ &isnull));
+ if (isnull)
+ elog(ERROR, "invalid NULL tgargs");
+ argstr = (char *) VARDATA(tgargs);
+ lentgargs = VARSIZE_ANY_EXHDR(tgargs);
+
+ p = argstr;
+ for (findx = 0; findx < tgnargs; findx++)
+ {
+ size_t tlen;
+
+ /* verify that the argument encoding is correct */
+ tlen = strlen(p);
+ if (p + tlen >= argstr + lentgargs)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid argument string (%s) for trigger \"%s\"",
+ argstr, NameStr(trigForm->tgname))));
+
+ list = lappend(list, new_string_object(p));
+
+ p += tlen + 1;
+ }
+ }
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{args:, }L", list); /* might be NIL */
+ append_format_string(tmp, ")");
+
+ append_object_object(trigger, "EXECUTE PROCEDURE %{function}s", tmp);
+
+ table_close(pg_trigger, AccessShareLock);
+
+ return trigger;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.).
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted.
+ *
+ * XXX maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I %{coltype}T %{compression}s %{default}s %{not_null}s
+ * %{collation}s
+ */
+ column = new_objtree_VA("%{name}I %{coltype}T", 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree("COMPRESSION");
+ if (coldef->compression)
+ append_string_object(tmp, "%{compression_method}I", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "%{compression_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "%{compression}s", tmp);
+
+ tmp = new_objtree("COLLATE");
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{collation}s", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "%{identity_column}s", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree("GENERATED ALWAYS AS");
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "%{generation_expr}s STORED", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "%{generated_column}s", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE OF deparsing.
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I WITH OPTIONS %{default}s %{not_null}s.
+ */
+ column = new_objtree_VA("%{name}I WITH OPTIONS", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /*
+ * Generated columns are not supported on typed tables, so we are done.
+ */
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparse_TableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing.
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *constr;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ constr = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(constr));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * Deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *oncommit;
+
+ oncommit = new_objtree("ON COMMIT");
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(oncommit, "%{on_commit_value}s", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(oncommit, "%{on_commit_value}s");
+ append_bool_object(oncommit, "present", false);
+ break;
+ }
+
+ return oncommit;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ set = new_objtree("");
+ optname = new_objtree("");
+
+ if (elem->defnamespace != NULL)
+ append_string_object(optname, "%{schema}I.", elem->defnamespace);
+
+ append_string_object(optname, "%{label}I", elem->defname);
+
+ append_object_object(set, "%{label}s", optname);
+
+ if (!is_reset)
+ append_string_object(set, " = %{value}L",
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *colset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER COLUMN %{column}I RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "ALTER COLUMN %{column}I RESET ";
+ else
+ fmt = "ALTER COLUMN %{column}I SET ";
+
+ colset = new_objtree_VA(fmt, 1, "column", ObjTypeString, subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(colset, "(%{options:, }s)", sets);
+
+ return colset;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *relset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "RESET ";
+ else
+ fmt = "SET ";
+
+ relset = new_objtree(fmt);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(relset, "(%{options:, }s)", sets);
+
+ return relset;
+}
+
+/*
+ * Deparse the inherits relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+/*
+ * Obtain the deparsed partition bound expression for the given table.
+ */
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* Output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, keyno == 0 ? "" : ", ");
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* Expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* If it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* Output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* Output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* Report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+}
+
+/*
+ * Deparse an IndexStmt.
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I ON
+ * %{table}D USING %{index_am}s (%{definition}s) %{with}s %{tablespace}s
+ * %{where_clause}s
+ */
+ indexStmt = new_objtree("CREATE");
+
+ append_string_object(indexStmt, "%{unique}s",
+ node->unique ? "UNIQUE" : "");
+
+ append_format_string(indexStmt, "INDEX");
+
+ append_string_object(indexStmt, "%{concurrently}s",
+ node->concurrent ? "CONCURRENTLY" : "");
+
+ append_string_object(indexStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(indexStmt, "%{name}I",
+ RelationGetRelationName(idxrel));
+
+ append_object_object(indexStmt, "ON %{table}D",
+ new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel)));
+
+ append_string_object(indexStmt, "USING %{index_am}s", index_am);
+
+ append_string_object(indexStmt, "(%{definition}s)", definition);
+
+ /* reloptions */
+ tmp = new_objtree("WITH");
+ if (reloptions)
+ append_string_object(tmp, "(%{opts}s)", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{with}s", tmp);
+
+ /* tablespace */
+ tmp = new_objtree("TABLESPACE");
+ if (tablespace)
+ append_string_object(tmp, "%{tablespace}s", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{tablespace}s", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree("WHERE");
+ if (whereClause)
+ append_string_object(tmp, "%{where}s", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{where_clause}s", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * Deparse a CreateStmt (CREATE TABLE).
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D [OF
+ * %{of_type}T | PARTITION OF %{parent_identity}D] %{table_elements}s
+ * %{inherits}s %{partition_by}s %{access_method}s %{with_clause}s
+ * %{on_commit}s %{tablespace}s
+ */
+ createStmt = new_objtree("CREATE");
+
+ append_string_object(createStmt, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ append_format_string(createStmt, "TABLE");
+
+ append_string_object(createStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "%{identity}D", tmp);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ /*
+ * Typed tables and partitions use a slightly different format string: we
+ * must not put table_elements with parents directly in the fmt string,
+ * because if there are no options the parens must not be emitted; and
+ * also, typed tables do not allow for inheritance.
+ */
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "OF %{of_type}T", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_format_string(createStmt, "PARTITION OF");
+
+ append_object_object(createStmt, "%{parent_identity}D",
+ elem->value.object);
+ }
+
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ {
+ tmp = new_objtree("");
+ append_bool_object(tmp, "present", false);
+ }
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "(%{table_elements:, }s)", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "(%{table_elements:, }s)", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree("INHERITS");
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "(%{parents:, }D)", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "(%{parents:, }D)");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{inherits}s", tmp);
+ }
+
+ tmp = new_objtree("TABLESPACE");
+ if (node->tablespacename)
+ append_string_object(tmp, "%{tablespace}I", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "%{tablespace}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{tablespace}s", tmp);
+
+ append_object_object(createStmt, "%{on_commit}s",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "%{partition_bound}s",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree("PARTITION BY");
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "%{definition}s", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "%{definition}s");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{partition_by}s", tmp);
+
+ /* USING clause */
+ tmp = new_objtree("USING");
+ if (node->accessMethod)
+ append_string_object(tmp, "%{access_method}I", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "%{access_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{access_method}s", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree("WITH");
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "(%{with:, }s)", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "%{with_clause}s", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_FunctionSet(VariableSetKind kind, char *name, char *value)
+{
+ ObjTree *r;
+
+ if (kind == VAR_RESET_ALL)
+ {
+ r = new_objtree("RESET ALL");
+ }
+ else if (value != NULL)
+ {
+ r = new_objtree_VA("SET %{set_name}I", 1,
+ "set_name", ObjTypeString, name);
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be
+ * quoted.
+ */
+ if (GetConfigOptionFlags(name, true) & GUC_LIST_QUOTE)
+ append_string_object(r, "TO %{set_value}s", value);
+ else
+ append_string_object(r, "TO %{set_value}L", value);
+ }
+ else
+ {
+ r = new_objtree("RESET");
+ append_string_object(r, "%{set_name}I", name);
+ }
+
+ return r;
+}
+
+/*
+ * deparse_CreateFunctionStmt
+ * Deparse a CreateFunctionStmt (CREATE FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
+static ObjTree *
+deparse_CreateFunction(Oid objectId, Node *parsetree)
+{
+ CreateFunctionStmt *node = (CreateFunctionStmt *) parsetree;
+ ObjTree *createFunc;
+ ObjTree *tmp;
+ Datum tmpdatum;
+ char *source;
+ char *probin;
+ List *params;
+ List *defaults;
+ List *sets = NIL;
+ List *types = NIL;
+ ListCell *cell;
+ ListCell *curdef;
+ ListCell *table_params = NULL;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ HeapTuple langTup;
+ Oid *typarray;
+ Oid *trftypes;
+ Form_pg_language langForm;
+ int i;
+ int typnum;
+ int ntypes;
+ bool isnull;
+ bool isfunction;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /* get the corresponding pg_language tuple */
+ langTup = SearchSysCache1(LANGOID, procForm->prolang);
+ if (!HeapTupleIsValid(langTup))
+ elog(ERROR, "cache lookup failure for language with OID %u",
+ procForm->prolang);
+ langForm = (Form_pg_language) GETSTRUCT(langTup);
+
+ /*
+ * Determine useful values for prosrc and probin. We cope with probin
+ * being either NULL or "-", but prosrc must have a valid value.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_prosrc, &isnull);
+ if (isnull)
+ elog(ERROR, "null prosrc in function with OID %u", objectId);
+ source = TextDatumGetCString(tmpdatum);
+
+ /* Determine a useful value for probin */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_probin, &isnull);
+ if (isnull)
+ probin = NULL;
+ else
+ {
+ probin = TextDatumGetCString(tmpdatum);
+ if (probin[0] == '\0' || strcmp(probin, "-") == 0)
+ probin = NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{or_replace}s FUNCTION %{signature}s RETURNS %{return_type}s
+ * LANGUAGE %{transform_type}s %{language}I %{window}s %{volatility}s
+ * %{parallel_safety}s %{leakproof}s %{strict}s %{security_definer}s
+ * %{cost}s %{rows}s %{support}s %{set_options: }s AS %{objfile}L,
+ * %{symbol}L
+ */
+ createFunc = new_objtree("CREATE");
+
+ append_string_object(createFunc, "%{or_replace}s",
+ node->replace ? "OR REPLACE" : "");
+
+ /*
+ * To construct the arguments array, extract the type OIDs from the
+ * function's pg_proc entry. If pronargs equals the parameter list length,
+ * there are no OUT parameters and thus we can extract the type OID from
+ * proargtypes; otherwise we need to decode proallargtypes, which is
+ * a bit more involved.
+ */
+ typarray = palloc(list_length(node->parameters) * sizeof(Oid));
+ if (list_length(node->parameters) > procForm->pronargs)
+ {
+ bool isnull;
+ Datum alltypes;
+ Datum *values;
+ bool *nulls;
+ int nelems;
+
+ alltypes = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proallargtypes, &isnull);
+ if (isnull)
+ elog(ERROR, "NULL proallargtypes, but more parameters than args");
+ deconstruct_array(DatumGetArrayTypeP(alltypes),
+ OIDOID, 4, 't', 'i',
+ &values, &nulls, &nelems);
+ if (nelems != list_length(node->parameters))
+ elog(ERROR, "mismatched proallargatypes");
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = values[i];
+ }
+ else
+ {
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = procForm->proargtypes.values[i];
+ }
+
+ /*
+ * If there are any default expressions, we read the deparsed expression as
+ * a list so that we can attach them to each argument.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proargdefaults, &isnull);
+ if (!isnull)
+ {
+ defaults = FunctionGetDefaults(DatumGetTextP(tmpdatum));
+ curdef = list_head(defaults);
+ }
+ else
+ {
+ defaults = NIL;
+ curdef = NULL;
+ }
+
+ /*
+ * Now iterate over each parameter in the parsetree to create the
+ * parameters array.
+ */
+ params = NIL;
+ typnum = 0;
+ foreach(cell, node->parameters)
+ {
+ FunctionParameter *param = (FunctionParameter *) lfirst(cell);
+ ObjTree *paramobj;
+ ObjTree *defaultval;
+ ObjTree *name;
+
+ /*
+ * A PARAM_TABLE parameter indicates end of input arguments; the
+ * following parameters are part of the return type. We ignore them
+ * here, but keep track of the current position in the list so that
+ * we can easily produce the return type below.
+ */
+ if (param->mode == FUNC_PARAM_TABLE)
+ {
+ table_params = cell;
+ break;
+ }
+
+ /*
+ * Verbose syntax for paramater: %{mode}s %{name}s %{type}T %{default}s
+ *
+ * Note that %{name}s is a string here, not an identifier; the reason
+ * for this is that an absent parameter name must produce an empty
+ * string, not "", which is what would happen if we were to use
+ * %{name}I here. So we add another level of indirection to allow us
+ * to inject a "present" parameter.
+ */
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{mode}s",
+ param->mode == FUNC_PARAM_IN ? "IN" :
+ param->mode == FUNC_PARAM_OUT ? "OUT" :
+ param->mode == FUNC_PARAM_INOUT ? "INOUT" :
+ param->mode == FUNC_PARAM_VARIADIC ? "VARIADIC" :
+ "IN");
+
+ /* optional wholesale suppression of "name" occurs here */
+
+ name = new_objtree("");
+ append_string_object(name, "%{name}I",
+ param->name ? param->name : "NULL");
+
+ append_bool_object(name, "present",
+ param->name ? true : false);
+
+ append_object_object(paramobj, "%{name}s", name);
+
+ defaultval = new_objtree("DEFAULT");
+ if (PointerIsValid(param->defexpr))
+ {
+ char *expr;
+
+ if (curdef == NULL)
+ elog(ERROR, "proargdefaults list too short");
+ expr = lfirst(curdef);
+
+ append_string_object(defaultval, "%{value}s", expr);
+ curdef = lnext(defaults, curdef);
+ }
+ else
+ append_bool_object(defaultval, "present", false);
+
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+
+ append_object_object(paramobj, "%{default}s", defaultval);
+
+ params = lappend(params, new_object_object(paramobj));
+ }
+
+ tmp = new_objtree_VA("%{identity}D", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{arguments:, }s", params);
+ append_format_string(tmp, ")");
+
+ isfunction = (procForm->prokind != PROKIND_PROCEDURE);
+
+ if (isfunction)
+ append_object_object(createFunc, "FUNCTION %{signature}s", tmp);
+ else
+ append_object_object(createFunc, "PROCEDURE %{signature}s", tmp);
+
+ /*
+ * A return type can adopt one of two forms: either a [SETOF] some_type, or
+ * a TABLE(list-of-types). We can tell the second form because we saw a
+ * table param above while scanning the argument list.
+ */
+ if (table_params == NULL)
+ {
+ tmp = new_objtree_VA("", 1,
+ "return_form", ObjTypeString, "plain");
+ append_string_object(tmp, "%{setof}s",
+ procForm->proretset ? "SETOF" : "");
+ append_object_object(tmp, "%{rettype}T",
+ new_objtree_for_type(procForm->prorettype, -1));
+ }
+ else
+ {
+ List *rettypes = NIL;
+ ObjTree *paramobj;
+
+ tmp = new_objtree_VA("TABLE", 1,
+ "return_form", ObjTypeString, "table");
+ for (; table_params != NULL; table_params = lnext(node->parameters, table_params))
+ {
+ FunctionParameter *param = lfirst(table_params);
+
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{name}I", param->name);
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+ rettypes = lappend(rettypes, new_object_object(paramobj));
+ }
+
+ append_array_object(tmp, "(%{rettypes:, }s)", rettypes);
+ }
+
+ if (isfunction)
+ append_object_object(createFunc, "RETURNS %{return_type}s", tmp);
+
+ /* TRANSFORM FOR TYPE */
+ tmp = new_objtree("TRANSFORM");
+
+ ntypes = get_func_trftypes(procTup, &trftypes);
+ for (i = 0; i < ntypes; i++)
+ {
+ tmp = new_objtree_VA("FOR TYPE %{type}T", 1,
+ "type", ObjTypeObject,
+ new_objtree_for_type(trftypes[i], -1));
+ types = lappend(types, tmp);
+ }
+
+ if (types)
+ append_array_object(tmp, "%{for_type:, }s", types);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{transform_type}s", tmp);
+
+ append_string_object(createFunc, "LANGUAGE %{language}I",
+ NameStr(langForm->lanname));
+
+ if (isfunction)
+ {
+ append_string_object(createFunc, "%{window}s",
+ procForm->prokind == PROKIND_WINDOW ? "WINDOW" : "");
+ append_string_object(createFunc, "%{volatility}s",
+ procForm->provolatile == PROVOLATILE_VOLATILE ?
+ "VOLATILE" :
+ procForm->provolatile == PROVOLATILE_STABLE ?
+ "STABLE" :
+ procForm->provolatile == PROVOLATILE_IMMUTABLE ?
+ "IMMUTABLE" : "INVALID VOLATILITY");
+
+ append_string_object(createFunc, "%{parallel_safety}s",
+ procForm->proparallel == PROPARALLEL_SAFE ?
+ "PARALLEL SAFE" :
+ procForm->proparallel == PROPARALLEL_RESTRICTED ?
+ "PARALLEL RESTRICTED" : "PARALLEL UNSAFE");
+
+ append_string_object(createFunc, "%{leakproof}s",
+ procForm->proleakproof ? "LEAKPROOF" : "");
+ append_string_object(createFunc, "%{strict}s",
+ procForm->proisstrict ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT");
+
+ append_string_object(createFunc, "%{security_definer}s",
+ procForm->prosecdef ?
+ "SECURITY DEFINER" : "SECURITY INVOKER");
+
+ append_object_object(createFunc, "%{cost}s",
+ new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ procForm->procost));
+
+ tmp = new_objtree("ROWS");
+ if (procForm->prorows == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n", procForm->prorows);
+ append_object_object(createFunc, "%{rows}s", tmp);
+
+ tmp = new_objtree("SUPPORT %{name}s");
+ if (procForm->prosupport)
+ {
+ Oid argtypes[1];
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{support}s", tmp);
+ }
+
+ foreach(cell, node->options)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ sets = lappend(sets, new_object_object(tmp));
+ }
+ }
+ append_array_object(createFunc, "%{set_options: }s", sets);
+
+ /* Add the function definition */
+ (void) SysCacheGetAttr(PROCOID, procTup, Anum_pg_proc_prosqlbody, &isnull);
+ if (procForm->prolang == SQLlanguageId && !isnull)
+ {
+ StringInfoData buf;
+
+ initStringInfo(&buf);
+ print_function_sqlbody(&buf, procTup);
+
+ append_string_object(createFunc, "%{definition}s", buf.data);
+ }
+ else if (probin == NULL)
+ {
+ append_string_object(createFunc, "AS %{definition}L",
+ source);
+ }
+ else
+ {
+ append_string_object(createFunc, "AS %{objfile}L", probin);
+ append_string_object(createFunc, ", %{symbol}L", source);
+ }
+
+ ReleaseSysCache(langTup);
+ ReleaseSysCache(procTup);
+
+ return createFunc;
+}
+
+/*
+ * Deparse a AlterFunctionStmt (ALTER FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the alter command.
+ */
+static ObjTree *
+deparse_AlterFunction(Oid objectId, Node *parsetree)
+{
+ AlterFunctionStmt *node = (AlterFunctionStmt *) parsetree;
+ ObjTree *alterFunc;
+ ObjTree *sign;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ List *params;
+ List *elems = NIL;
+ ListCell *cell;
+ int i;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER FUNCTION %{signature}s %{definition: }s
+ */
+ alterFunc = new_objtree_VA("ALTER FUNCTION", 0);
+
+ params = NIL;
+
+ /*
+ * ALTER FUNCTION does not change signature so we can use catalog
+ * to get input type Oids.
+ */
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *tmp = new_objtree_VA("%{type}T", 0);
+
+ append_object_object(tmp, "type",
+ new_objtree_for_type(procForm->proargtypes.values[i], -1));
+ params = lappend(params, new_object_object(tmp));
+ }
+
+ sign = new_objtree("");
+
+ append_object_object(sign, "%{identity}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+ append_array_object(sign, "(%{arguments:, }s)", params);
+
+ append_object_object(alterFunc, "%{signature}s", sign);
+
+ foreach(cell, node->actions)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "volatility") == 0)
+ {
+ tmp = new_objtree_VA(strVal(defel->arg), 0);
+ }
+ else if (strcmp(defel->defname, "strict") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT", 0);
+ }
+ else if (strcmp(defel->defname, "security") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "SECURITY DEFINER" : "SECURITY INVOKER", 0);
+ }
+ else if (strcmp(defel->defname, "leakproof") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "LEAKPROOF" : "NOT LEAKPROOF", 0);
+ }
+ else if (strcmp(defel->defname, "cost") == 0)
+ {
+ tmp = new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "rows") == 0)
+ {
+ tmp = new_objtree_VA("ROWS", 0);
+ if (defGetNumeric(defel) == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n",
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ }
+ else if (strcmp(defel->defname, "support") == 0)
+ {
+ Oid argtypes[1];
+
+ tmp = new_objtree("SUPPORT");
+
+ Assert(procForm->prosupport);
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else if (strcmp(defel->defname, "parallel") == 0)
+ {
+ char *fmt = psprintf("PARALLEL %s", strVal(defel->arg));
+ tmp = new_objtree(fmt);
+ }
+
+ elems = lappend(elems, new_object_object(tmp));
+ }
+
+ append_array_object(alterFunc, "%{definition: }s", elems);
+
+ ReleaseSysCache(procTup);
+
+ return alterFunc;
+}
+
+/*
+ * Deparse a AlterOwnerStmt (ALTER ... OWNER TO ...).
+ */
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+
+ ownerStmt = new_objtree_VA(fmt, 2,
+ "identity", ObjTypeString,
+ getObjectIdentity(&address, false),
+ "newowner", ObjTypeString,
+ get_rolespec_name(node->newowner));
+
+ return ownerStmt;
+}
+
+/*
+ * Deparse a AlterOperatorStmt (ALTER OPERATOR ... SET ...).
+ */
+static ObjTree *
+deparse_AlterOperatorStmt(Oid objectId, Node *parsetree)
+{
+ HeapTuple oprTup;
+ AlterOperatorStmt *node = (AlterOperatorStmt *) parsetree;
+ ObjTree *alterop;
+ Form_pg_operator oprForm;
+ ListCell *cell;
+ List *list = NIL;
+
+ oprTup = SearchSysCache1(OPEROID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(oprTup))
+ elog(ERROR, "cache lookup failed for operator with OID %u", objectId);
+ oprForm = (Form_pg_operator) GETSTRUCT(oprTup);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER OPERATOR %{identity}O (%{left_type}T, %{right_type}T) SET
+ * (%{elems:, }s)
+ */
+ alterop = new_objtree_VA("ALTER OPERATOR %{identity}O", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(oprForm->oprnamespace,
+ NameStr(oprForm->oprname)));
+
+ /* LEFTARG */
+ if (OidIsValid(oprForm->oprleft))
+ append_object_object(alterop, "(%{left_type}T",
+ new_objtree_for_type(oprForm->oprleft, -1));
+ else
+ append_string_object(alterop, "(%{left_type}s", "NONE");
+
+ /* RIGHTARG */
+ Assert(OidIsValid(oprForm->oprleft));
+ append_object_object(alterop, ", %{right_type}T)",
+ new_objtree_for_type(oprForm->oprright, -1));
+
+ foreach(cell, node->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(elem->defname, "restrict") == 0)
+ {
+ tmp = new_objtree_VA("RESTRICT=", 1,
+ "clause", ObjTypeString, "restrict");
+ if (OidIsValid(oprForm->oprrest))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprrest));
+ else
+ append_string_object(tmp, "%{procedure}s", "NONE");
+ }
+ else if (strcmp(elem->defname, "join") == 0)
+ {
+ tmp = new_objtree_VA("JOIN=", 1,
+ "clause", ObjTypeString, "join");
+ if (OidIsValid(oprForm->oprjoin))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprjoin));
+ else
+ append_string_object(tmp, "%{procedure}s", "NONE");
+ }
+
+ Assert(tmp);
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ append_array_object(alterop, "SET (%{elems:, }s)", list);
+
+ ReleaseSysCache(oprTup);
+
+ return alterop;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Deparse a RenameStmt.
+ */
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+ char *fmtstr;
+ const char *objtype;
+ Relation relation;
+ Oid schemaId;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_TABLE:
+ case OBJECT_INDEX:
+ case OBJECT_SEQUENCE:
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER %s %{if_exists}s %{identity}D RENAME TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+ renameStmt = new_objtree_VA(fmtstr, 0);
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+ append_string_object(renameStmt, "RENAME TO %{newname}I",
+ node->newname);
+ relation_close(relation, AccessShareLock);
+ break;
+
+ case OBJECT_ATTRIBUTE:
+ case OBJECT_COLUMN:
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER objtype %{identity}D RENAME ATTRIBUTE
+ * %{colname}I TO %{newname}I %{cascade}s
+ *
+ * Normal types: ALTER objtype %{if_exists}s %%{identity}D RENAME
+ * COLUMN %{colname}I TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ renameStmt = new_objtree(fmtstr);
+
+ /* Composite types do not support IF EXISTS */
+ if (node->renameType == OBJECT_COLUMN)
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+
+ fmtstr = psprintf("RENAME %s %%{colname}I", objtype);
+
+ append_string_object(renameStmt, fmtstr, node->subname);
+ append_string_object(renameStmt, "TO %{newname}I", node->newname);
+
+ if (node->renameType == OBJECT_ATTRIBUTE)
+ append_object_object(renameStmt, "%{cascade}s",
+ new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool,
+ node->behavior == DROP_CASCADE));
+
+ relation_close(relation, AccessShareLock);
+
+ break;
+ case OBJECT_SCHEMA:
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 1,
+ "identity", ObjTypeString, node->subname,
+ "newname", ObjTypeString, node->newname);
+ break;
+ case OBJECT_TRIGGER:
+ {
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ Relation pg_trigger;
+
+ pg_trigger = relation_open(TriggerRelationId, AccessShareLock);
+ trigTup = get_catalog_object_by_oid(pg_trigger, get_object_attnum_oid(address.classId), address.objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ renameStmt = new_objtree_VA("ALTER TRIGGER %{triggername}I ON %{identity}D RENAME TO %{newname}I",
+ 3,
+ "triggername", ObjTypeString, node->subname,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid),
+ "newname", ObjTypeString, node->newname);
+
+ relation_close(pg_trigger, AccessShareLock);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ return renameStmt;
+}
+
+/*
+ * Deparse the sequence CACHE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cache;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ cache = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(cache);
+}
+
+/*
+ * Deparse the sequence CYCLE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cycle;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ cycle = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(cycle);
+}
+
+/*
+ * Deparse the sequence INCREMENT BY option.
+ */
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *incrementalby;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ incrementalby = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(incrementalby);
+}
+
+/*
+ * Deparse the sequence MINVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *minvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ minvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(minvalue);
+}
+
+/*
+ * Deparse the sequence MAXVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *maxvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ maxvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(maxvalue);
+}
+
+/*
+ * Deparse the sequence START WITH option.
+ */
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *startwith;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ startwith = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(startwith);
+}
+
+/*
+ * Deparse the sequence RESTART option.
+ */
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *restart;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ restart = new_objtree_VA("RESTART %{value}s", 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(restart);
+}
+
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s
+ */
+ createSchema = new_objtree("CREATE SCHEMA");
+
+ append_string_object(createSchema, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(createSchema, "%{name}I", node->schemaname);
+
+ auth = new_objtree("AUTHORIZATION");
+ if (node->authrole)
+ append_string_object(auth, "%{authorization_role}I",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "%{authorization_role}I ");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "%{authorization}s", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/*
+ * Deparse a CreateSeqStmt.
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s SEQUENCE %{identity}D
+ */
+ createSeq = new_objtree("CREATE");
+
+ append_string_object(createSeq, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "SEQUENCE %{identity}D", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * Deparse an AlterSeqStmt.
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq = new_objtree("ALTER SEQUENCE");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "%{identity}D", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+/*
+ * Deparse all the collected subcommands and return an ObjTree representing the
+ * alter command.
+ */
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER reltype %{identity}D %{subcmds:, }s
+ */
+ fmtstr = psprintf("ALTER %s", reltype);
+ alterTableStmt = new_objtree(fmtstr);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "%{identity}D", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+
+ /*
+ * Verbose syntax
+ *
+ * ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX
+ * %index_name}I %{deferrable}s %{init_deferred}s
+ */
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I",
+ 2,
+ "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+
+ append_string_object(tmp, "%{constraint_type}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "USING INDEX %{index_name}I",
+ RelationGetRelationName(idx));
+ append_string_object(tmp, "%{deferrable}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "%{init_deferred}s", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 2,
+ "type", ObjTypeString, "drop default",
+ "column", ObjTypeString, subcmd->name);
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT",
+ 2,
+ "type", ObjTypeString, "set default",
+ "column", ObjTypeString, subcmd->name);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "%{definition}s",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 2,
+ "type", ObjTypeString, "drop not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 2,
+ "type", ObjTypeString, "set not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION",
+ 2,
+ "type", ObjTypeString, "drop expression",
+ "column", ObjTypeString, subcmd->name);
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS": "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3,
+ "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3,
+ "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* If no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 4,
+ "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "deferrable", ObjTypeString,
+ c->deferrable ? "DEFERRABLE" : "NOT DEFERRABLE",
+ "init_deferred", ObjTypeString,
+ c->initdeferred ? "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s ATTRIBUTE %{cascade}s
+ *
+ * Normal types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s COLUMN %{using}s
+ */
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE",
+ istype ? "ATTRIBUTE" : "COLUMN");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* Add the TYPE clause */
+ append_object_object(tmp, "%{datatype}T",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* Add a COLLATE clause, if needed */
+ tmp2 = new_objtree("COLLATE");
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{collation}s", tmp2);
+
+ /* If not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree("USING");
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "%{expression}s",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{using}s", tmp2);
+ }
+
+ /* If it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree("CASCADE");
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I", 2,
+ "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T", 2,
+ "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF", 1,
+ "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "%{ident}s", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "%{ident}s", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "%{ident}s", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "%{ident}s", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D %{partition_bound}s",
+ 3,
+ "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId),
+ "partition_bound", ObjTypeString,
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2,
+ "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2,
+ "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2,
+ "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I",
+ 2,
+ "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "%{definition}s", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY",
+ 2,
+ "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name);
+
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS" : "");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "%{subcmds:, }s", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Deparse a CREATE OPERATOR CLASS command.
+ */
+static ObjTree *
+deparse_CreateOpClassStmt(CollectedCommand *cmd)
+{
+ Oid opcoid = cmd->d.createopc.address.objectId;
+ HeapTuple opcTup;
+ HeapTuple opfTup;
+ Form_pg_opfamily opfForm;
+ Form_pg_opclass opcForm;
+ ObjTree *stmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ opcTup = SearchSysCache1(CLAOID, ObjectIdGetDatum(opcoid));
+ if (!HeapTupleIsValid(opcTup))
+ elog(ERROR, "cache lookup failed for opclass with OID %u", opcoid);
+ opcForm = (Form_pg_opclass) GETSTRUCT(opcTup);
+
+ opfTup = SearchSysCache1(OPFAMILYOID, opcForm->opcfamily);
+ if (!HeapTupleIsValid(opfTup))
+ elog(ERROR, "cache lookup failed for operator family with OID %u", opcForm->opcfamily);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(opfTup);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE OPERATOR CLASS %{identity}D %{default}s FOR TYPE %{type}T USING
+ * %{amname}I %{opfamily}s AS %{items:, }s
+ */
+
+ stmt = new_objtree_VA("CREATE OPERATOR CLASS %{identity}D", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opcForm->opcnamespace,
+ NameStr(opcForm->opcname)));
+
+ /* Add the DEFAULT clause */
+ append_string_object(stmt, "%{default}s",
+ opcForm->opcdefault ? "DEFAULT" : "");
+
+ /* Add the FOR TYPE clause */
+ append_object_object(stmt, "FOR TYPE %{type}T",
+ new_objtree_for_type(opcForm->opcintype, -1));
+
+ /* Add the USING clause */
+ append_string_object(stmt, "USING %{amname}I", get_am_name(opcForm->opcmethod));
+
+ /*
+ * Add the FAMILY clause; but if it has the same name and namespace as the
+ * opclass, then have it expand to empty, because it would cause a failure
+ * if the opfamily was created internally.
+ */
+ tmp = new_objtree_VA("FAMILY %{opfamily}D", 1,
+ "opfamily", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)));
+
+ if (strcmp(NameStr(opfForm->opfname), NameStr(opcForm->opcname)) == 0 &&
+ opfForm->opfnamespace == opcForm->opcnamespace)
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(stmt, "%{opfamily}s", tmp);
+
+ /*
+ * Add the initial item list. Note we always add the STORAGE clause, even
+ * when it is implicit in the original command.
+ */
+ tmp = new_objtree("STORAGE");
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(opcForm->opckeytype != InvalidOid ?
+ opcForm->opckeytype : opcForm->opcintype,
+ -1));
+ list = list_make1(new_object_object(tmp));
+
+ /* Add the declared operators */
+ foreach(cell, cmd->d.createopc.operators)
+ {
+ OpFamilyMember *oper = lfirst(cell);
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("OPERATOR %{num}n %{operator}O(%{ltype}T, %{rtype}T)",
+ 4,
+ "num", ObjTypeInteger, oper->number,
+ "operator", ObjTypeObject,
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oper->object),
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(oper->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(oper->righttype, -1));
+
+ /* Add the FOR SEARCH / FOR ORDER BY clause */
+ if (oper->sortfamily == InvalidOid)
+ append_string_object(tmp, "%{purpose}s", "FOR SEARCH");
+ else
+ {
+ ObjTree *tmp2;
+
+ tmp2 = new_objtree_VA("FOR ORDER BY %{opfamily}D", 0);
+ append_object_object(tmp2, "opfamily",
+ new_objtree_for_qualname_id(OperatorFamilyRelationId,
+ oper->sortfamily));
+ append_object_object(tmp, "%{purpose}s", tmp2);
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ /* Add the declared support functions */
+ foreach(cell, cmd->d.createopc.procedures)
+ {
+ OpFamilyMember *proc = lfirst(cell);
+ ObjTree *tmp;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ Oid *proargtypes;
+ List *arglist;
+ int i;
+
+ procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(proc->object));
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failed for procedure %u", proc->object);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ tmp = new_objtree_VA("FUNCTION %{num}n (%{ltype}T, %{rtype}T) %{function}D",
+ 4,
+ "num", ObjTypeInteger, proc->number,
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(proc->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(proc->righttype, -1),
+ "function", ObjTypeObject,
+ new_objtree_for_qualname(procForm->pronamespace,
+ NameStr(procForm->proname)));
+
+ proargtypes = procForm->proargtypes.values;
+ arglist = NIL;
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *arg;
+
+ arg = new_objtree_for_type(proargtypes[i], -1);
+ arglist = lappend(arglist, new_object_object(arg));
+ }
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{argtypes:, }T", arglist);
+ append_format_string(tmp, ")");
+
+ ReleaseSysCache(procTup);
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ append_array_object(stmt, "AS %{items:, }s", list);
+
+ ReleaseSysCache(opfTup);
+ ReleaseSysCache(opcTup);
+
+ return stmt;
+}
+
+/*
+ * Deparse a CreateTrigStmt (CREATE OPERATOR FAMILY)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateOpFamily(Oid objectId, Node *parsetree)
+{
+ HeapTuple opfTup;
+ HeapTuple amTup;
+ Form_pg_opfamily opfForm;
+ Form_pg_am amForm;
+ ObjTree *copfStmt;
+
+ opfTup = SearchSysCache1(OPFAMILYOID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(opfTup))
+ elog(ERROR, "cache lookup failed for operator family with OID %u", objectId);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(opfTup);
+
+ amTup = SearchSysCache1(AMOID, ObjectIdGetDatum(opfForm->opfmethod));
+ if (!HeapTupleIsValid(amTup))
+ elog(ERROR, "cache lookup failed for access method %u",
+ opfForm->opfmethod);
+ amForm = (Form_pg_am) GETSTRUCT(amTup);
+
+ copfStmt = new_objtree_VA("CREATE OPERATOR FAMILY %{identity}D USING %{amname}I",
+ 2,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)),
+ "amname", ObjTypeString, NameStr(amForm->amname));
+
+ ReleaseSysCache(amTup);
+ ReleaseSysCache(opfTup);
+
+ return copfStmt;
+}
+
+/*
+ * Deparse an ALTER OPERATOR FAMILY ADD/DROP command.
+ */
+static ObjTree *
+deparse_AlterOpFamily(CollectedCommand *cmd)
+{
+ ObjTree *alterOpFam;
+ AlterOpFamilyStmt *stmt = (AlterOpFamilyStmt *) cmd->parsetree;
+ HeapTuple ftp;
+ Form_pg_opfamily opfForm;
+ List *list;
+ ListCell *cell;
+
+ ftp = SearchSysCache1(OPFAMILYOID,
+ ObjectIdGetDatum(cmd->d.opfam.address.objectId));
+ if (!HeapTupleIsValid(ftp))
+ elog(ERROR, "cache lookup failed for operator family %u",
+ cmd->d.opfam.address.objectId);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(ftp);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER OPERATOR FAMILY %{identity}D USING %{amname}I ADD/DROP
+ * %{items:, }s
+ */
+ alterOpFam = new_objtree_VA("ALTER OPERATOR FAMILY %{identity}D "
+ "USING %{amname}I", 2,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)),
+ "amname", ObjTypeString, stmt->amname);
+
+ list = NIL;
+ foreach(cell, cmd->d.opfam.operators)
+ {
+ OpFamilyMember *oper = lfirst(cell);
+ ObjTree *tmp;
+
+ /*
+ * Verbose syntax
+ *
+ * OPERATOR %{num}n %{operator}O(%{ltype}T, %{rtype}T) %{purpose}s
+ */
+ tmp = new_objtree_VA("OPERATOR %{num}n", 1,
+ "num", ObjTypeInteger, oper->number);
+
+ /* Add the operator name; the DROP case doesn't have this */
+ if (!stmt->isDrop)
+ {
+ append_object_object(tmp, "%{operator}O",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oper->object));
+ }
+
+ /* Add the types */
+ append_object_object(tmp, "(%{ltype}T,",
+ new_objtree_for_type(oper->lefttype, -1));
+ append_object_object(tmp, "%{rtype}T)",
+ new_objtree_for_type(oper->righttype, -1));
+
+ /* Add the FOR SEARCH / FOR ORDER BY clause; not in the DROP case */
+ if (!stmt->isDrop)
+ {
+ if (oper->sortfamily == InvalidOid)
+ append_string_object(tmp, "%{purpose}s", "FOR SEARCH");
+ else
+ {
+ ObjTree *tmp2;
+
+ tmp2 = new_objtree_VA("FOR ORDER BY", 0);
+ append_object_object(tmp2, "%{opfamily}D",
+ new_objtree_for_qualname_id(OperatorFamilyRelationId,
+ oper->sortfamily));
+ append_object_object(tmp, "%{purpose}s", tmp2);
+ }
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ foreach(cell, cmd->d.opfam.procedures)
+ {
+ OpFamilyMember *proc = lfirst(cell);
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("FUNCTION %{num}n (%{ltype}T, %{rtype}T)", 3,
+ "num", ObjTypeInteger, proc->number,
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(proc->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(proc->righttype, -1));
+
+ /* Add the function name and arg types; the DROP case doesn't have this */
+ if (!stmt->isDrop)
+ {
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ Oid *proargtypes;
+ List *arglist;
+ int i;
+
+ procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(proc->object));
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failed for procedure %u", proc->object);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ proargtypes = procForm->proargtypes.values;
+ arglist = NIL;
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *arg;
+
+ arg = new_objtree_for_type(proargtypes[i], -1);
+ arglist = lappend(arglist, new_object_object(arg));
+ }
+
+ append_object_object(tmp, "%{function}D",
+ new_objtree_for_qualname(procForm->pronamespace,
+ NameStr(procForm->proname)));
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{argtypes:, }T", arglist);
+ append_format_string(tmp, ")");
+
+ ReleaseSysCache(procTup);
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ if (!stmt->isDrop)
+ append_format_string(alterOpFam, "ADD");
+ else
+ append_format_string(alterOpFam, "DROP");
+
+ append_array_object(alterOpFam, "%{items:, }s", list);
+
+ ReleaseSysCache(ftp);
+
+ return alterOpFam;
+}
+
+/*
+ * Deparse a DefineStmt (CREATE OPERATOR)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_DefineStmt_Operator(Oid objectId, DefineStmt *define)
+{
+ HeapTuple oprTup;
+ ObjTree *stmt;
+ ObjTree *tmp;
+ List *list;
+ Form_pg_operator oprForm;
+
+ oprTup = SearchSysCache1(OPEROID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(oprTup))
+ elog(ERROR, "cache lookup failed for operator with OID %u", objectId);
+ oprForm = (Form_pg_operator) GETSTRUCT(oprTup);
+
+ stmt = new_objtree_VA("CREATE OPERATOR %{identity}O", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(oprForm->oprnamespace,
+ NameStr(oprForm->oprname)));
+
+ /* Add the definition clause */
+ list = NIL;
+
+ /* PROCEDURE */
+ tmp = new_objtree_VA("PROCEDURE=%{procedure}D", 2,
+ "clause", ObjTypeString, "procedure",
+ "procedure", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprcode));
+ list = lappend(list, new_object_object(tmp));
+
+ /* LEFTARG */
+ tmp = new_objtree_VA("LEFTARG=", 1,
+ "clause", ObjTypeString, "leftarg");
+ if (OidIsValid(oprForm->oprleft))
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(oprForm->oprleft, -1));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* RIGHTARG */
+ tmp = new_objtree_VA("RIGHTARG=", 1,
+ "clause", ObjTypeString, "rightarg");
+ if (OidIsValid(oprForm->oprright))
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(oprForm->oprright, -1));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* COMMUTATOR */
+ tmp = new_objtree_VA("COMMUTATOR=", 1,
+ "clause", ObjTypeString, "commutator");
+ if (OidIsValid(oprForm->oprcom))
+ append_object_object(tmp, "%{oper}D",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oprForm->oprcom));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* NEGATOR */
+ tmp = new_objtree_VA("NEGATOR=", 1,
+ "clause", ObjTypeString, "negator");
+ if (OidIsValid(oprForm->oprnegate))
+ append_object_object(tmp, "%{oper}D",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oprForm->oprnegate));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* RESTRICT */
+ tmp = new_objtree_VA("RESTRICT=", 1,
+ "clause", ObjTypeString, "restrict");
+ if (OidIsValid(oprForm->oprrest))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprrest));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* JOIN */
+ tmp = new_objtree_VA("JOIN=", 1,
+ "clause", ObjTypeString, "join");
+ if (OidIsValid(oprForm->oprjoin))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprjoin));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* MERGES */
+ tmp = new_objtree_VA("MERGES", 1,
+ "clause", ObjTypeString, "merges");
+ if (!oprForm->oprcanmerge)
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* HASHES */
+ tmp = new_objtree_VA("HASHES", 1,
+ "clause", ObjTypeString, "hashes");
+ if (!oprForm->oprcanhash)
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ append_array_object(stmt, "(%{elems:, }s)", list);
+
+ ReleaseSysCache(oprTup);
+
+ return stmt;
+}
+
+/*
+ * Deparse a DefineStmt.
+ */
+static ObjTree *
+deparse_DefineStmt(Oid objectId, Node *parsetree, ObjectAddress secondaryObj)
+{
+ DefineStmt *define = (DefineStmt *) parsetree;
+ ObjTree *defStmt;
+
+ switch (define->kind)
+ {
+ case OBJECT_OPERATOR:
+ defStmt = deparse_DefineStmt_Operator(objectId, define);
+ break;
+
+ default:
+ elog(ERROR, "unsupported object kind");
+ return NULL;
+ }
+
+ return defStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateTrigStmt:
+ command = deparse_CreateTrigStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_DefineStmt:
+ command = deparse_DefineStmt(objectId, parsetree,
+ cmd->d.simple.secondaryObject);
+ break;
+
+ case T_CreateOpFamilyStmt:
+ command = deparse_CreateOpFamily(objectId, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ case T_CreateFunctionStmt:
+ command = deparse_CreateFunction(objectId, parsetree);
+ break;
+
+ case T_AlterFunctionStmt:
+ command = deparse_AlterFunction(objectId, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_AlterOperatorStmt:
+ command = deparse_AlterOperatorStmt(objectId, parsetree);
+ break;
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+/*
+ * Handle deparsing of DROP commands.
+ */
+char *
+deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ char *identity = (char *) objidentity;
+ char *fmt;
+ ObjTree *stmt, *stmt2;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+
+ fmt = psprintf("DROP %s IF EXISTS %%{objidentity}s", objecttype);
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString, identity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+
+ append_object_object(stmt, "%{cascade}s", stmt2);
+
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+
+ return command;
+}
+
+/*
+ * Workhorse to deparse a CollectedCommand.
+ */
+char *
+deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ verbose = verbose_mode;
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ case SCT_AlterOpFamily:
+ tree = deparse_AlterOpFamily(cmd);
+ break;
+ case SCT_CreateOpClass:
+ tree = deparse_CreateOpClassStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd, true);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..d64719b
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,760 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo buf, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvBool, an error is raised. If it doesn't
+ * exist, tv_absent is returned; otherwise return the actual trivalue.
+ */
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised unless missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo buf)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * Found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /*
+ * Validate that we got an array if the format string specified one.
+ * And finally print out the data
+ */
+ if (is_array)
+ expand_jsonb_array(buf, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(buf, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and may contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * Expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal.
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* Easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar = dqnextchar % (sizeof(dqsuffixes) - 1);
+ }
+ /* Add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity.
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo buf, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(buf, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(buf, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(buf, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(buf, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(buf, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(buf, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(buf, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(buf, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ bool first = true;
+ StringInfoData arrayelem;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if (!JsonContainerIsArray(container))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ initStringInfo(&arrayelem);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ if (type == WJB_ELEM)
+ {
+ resetStringInfo(&arrayelem);
+
+ if (expand_one_jsonb_element(&arrayelem, param, &v, specifier, NULL))
+ {
+ if (!first)
+ appendStringInfoString(buf, arraysep);
+
+ appendBinaryStringInfo(buf, arrayelem.data, arrayelem.len);
+ first = false;
+ }
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+/*
+ * Workhorse for ddl_deparse_expand_command.
+ */
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo buf = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(buf);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, buf);
+
+ return buf->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable.
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index b0b2118..289a5de 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1700,6 +1700,39 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ /* Open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7fbee0c..0da73bb 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5193,11 +5193,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
cur_pass, context);
Assert(cmd != NULL);
if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- ATExecAttachPartition(wqueue, rel, (PartitionCmd *) cmd->def,
- context);
+ address = ATExecAttachPartition(wqueue, rel,
+ (PartitionCmd *) cmd->def,
+ context);
else
- ATExecAttachPartitionIdx(wqueue, rel,
- ((PartitionCmd *) cmd->def)->name);
+ address = ATExecAttachPartitionIdx(wqueue, rel,
+ ((PartitionCmd *) cmd->def)->name);
break;
case AT_DetachPartition:
cmd = ATParseTransformCmd(wqueue, tab, rel, cmd, false, lockmode,
@@ -5205,12 +5206,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
Assert(cmd != NULL);
/* ATPrepCmd ensures it must be a table */
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
- ATExecDetachPartition(wqueue, tab, rel,
- ((PartitionCmd *) cmd->def)->name,
- ((PartitionCmd *) cmd->def)->concurrent);
+ address = ATExecDetachPartition(wqueue, tab, rel,
+ ((PartitionCmd *) cmd->def)->name,
+ ((PartitionCmd *) cmd->def)->concurrent);
break;
case AT_DetachPartitionFinalize:
- ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
+ address = ATExecDetachPartitionFinalize(rel, ((PartitionCmd *) cmd->def)->name);
break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
diff --git a/src/backend/utils/adt/format_type.c b/src/backend/utils/adt/format_type.c
index 060fd7e..6af2257 100644
--- a/src/backend/utils/adt/format_type.c
+++ b/src/backend/utils/adt/format_type.c
@@ -27,7 +27,6 @@
#include "utils/numeric.h"
#include "utils/syscache.h"
-static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
/*
@@ -367,7 +366,7 @@ format_type_with_typemod(Oid type_oid, int32 typemod)
/*
* Add typmod decoration to the basic type name
*/
-static char *
+char *
printTypmod(const char *typname, int32 typmod, Oid typmodout)
{
char *res;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index d575aa0..95e6780 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -358,7 +358,6 @@ static int print_function_arguments(StringInfo buf, HeapTuple proctup,
bool print_table_args, bool print_defaults);
static void print_function_rettype(StringInfo buf, HeapTuple proctup);
static void print_function_trftypes(StringInfo buf, HeapTuple proctup);
-static void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
static void set_rtable_names(deparse_namespace *dpns, List *parent_namespaces,
Bitmapset *rels_used);
static void set_deparse_for_query(deparse_namespace *dpns, Query *query,
@@ -488,22 +487,15 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
static char *generate_relation_name(Oid relid, List *namespaces);
static char *generate_qualified_relation_name(Oid relid);
-static char *generate_function_name(Oid funcid, int nargs,
- List *argnames, Oid *argtypes,
- bool has_variadic, bool *use_variadic_p,
- ParseExprKind special_exprkind);
static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1025,65 +1017,12 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
if (!isnull)
{
Node *qual;
- char relkind;
- deparse_context context;
- deparse_namespace dpns;
- RangeTblEntry *oldrte;
- RangeTblEntry *newrte;
-
- appendStringInfoString(&buf, "WHEN (");
+ char *qualstr;
qual = stringToNode(TextDatumGetCString(value));
+ qualstr = pg_get_trigger_whenclause(trigrec, qual, pretty);
- relkind = get_rel_relkind(trigrec->tgrelid);
-
- /* Build minimal OLD and NEW RTEs for the rel */
- oldrte = makeNode(RangeTblEntry);
- oldrte->rtekind = RTE_RELATION;
- oldrte->relid = trigrec->tgrelid;
- oldrte->relkind = relkind;
- oldrte->rellockmode = AccessShareLock;
- oldrte->alias = makeAlias("old", NIL);
- oldrte->eref = oldrte->alias;
- oldrte->lateral = false;
- oldrte->inh = false;
- oldrte->inFromCl = true;
-
- newrte = makeNode(RangeTblEntry);
- newrte->rtekind = RTE_RELATION;
- newrte->relid = trigrec->tgrelid;
- newrte->relkind = relkind;
- newrte->rellockmode = AccessShareLock;
- newrte->alias = makeAlias("new", NIL);
- newrte->eref = newrte->alias;
- newrte->lateral = false;
- newrte->inh = false;
- newrte->inFromCl = true;
-
- /* Build two-element rtable */
- memset(&dpns, 0, sizeof(dpns));
- dpns.rtable = list_make2(oldrte, newrte);
- dpns.subplans = NIL;
- dpns.ctes = NIL;
- dpns.appendrels = NULL;
- set_rtable_names(&dpns, NIL, NULL);
- set_simple_column_names(&dpns);
-
- /* Set up context with one-deep namespace stack */
- context.buf = &buf;
- context.namespaces = list_make1(&dpns);
- context.windowClause = NIL;
- context.windowTList = NIL;
- context.varprefix = true;
- context.prettyFlags = GET_PRETTY_FLAGS(pretty);
- context.wrapColumn = WRAP_COLUMN_DEFAULT;
- context.indentLevel = PRETTYINDENT_STD;
- context.special_exprkind = EXPR_KIND_NONE;
- context.appendparents = NULL;
-
- get_rule_expr(qual, &context, false);
-
- appendStringInfoString(&buf, ") ");
+ appendStringInfo(&buf, "WHEN (%s) ", qualstr);
}
appendStringInfo(&buf, "EXECUTE FUNCTION %s(",
@@ -1124,6 +1063,64 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
return buf.data;
}
+char *
+pg_get_trigger_whenclause(Form_pg_trigger trigrec, Node *whenClause, bool pretty)
+{
+ StringInfoData buf;
+ char relkind;
+ deparse_context context;
+ deparse_namespace dpns;
+ RangeTblEntry *oldrte;
+ RangeTblEntry *newrte;
+
+ initStringInfo(&buf);
+
+ relkind = get_rel_relkind(trigrec->tgrelid);
+
+ /* Build minimal OLD and NEW RTEs for the rel */
+ oldrte = makeNode(RangeTblEntry);
+ oldrte->rtekind = RTE_RELATION;
+ oldrte->relid = trigrec->tgrelid;
+ oldrte->relkind = relkind;
+ oldrte->alias = makeAlias("old", NIL);
+ oldrte->eref = oldrte->alias;
+ oldrte->lateral = false;
+ oldrte->inh = false;
+ oldrte->inFromCl = true;
+
+ newrte = makeNode(RangeTblEntry);
+ newrte->rtekind = RTE_RELATION;
+ newrte->relid = trigrec->tgrelid;
+ newrte->relkind = relkind;
+ newrte->alias = makeAlias("new", NIL);
+ newrte->eref = newrte->alias;
+ newrte->lateral = false;
+ newrte->inh = false;
+ newrte->inFromCl = true;
+
+ /* Build two-element rtable */
+ memset(&dpns, 0, sizeof(dpns));
+ dpns.rtable = list_make2(oldrte, newrte);
+ dpns.ctes = NIL;
+ set_rtable_names(&dpns, NIL, NULL);
+ set_simple_column_names(&dpns);
+
+ /* Set up context with one-deep namespace stack */
+ context.buf = &buf;
+ context.namespaces = list_make1(&dpns);
+ context.windowClause = NIL;
+ context.windowTList = NIL;
+ context.varprefix = true;
+ context.prettyFlags = pretty ? PRETTYFLAG_PAREN | PRETTYFLAG_INDENT : PRETTYFLAG_INDENT;
+ context.wrapColumn = WRAP_COLUMN_DEFAULT;
+ context.indentLevel = PRETTYINDENT_STD;
+ context.special_exprkind = EXPR_KIND_NONE;
+
+ get_rule_expr(whenClause, &context, false);
+
+ return buf.data;
+}
+
/* ----------
* pg_get_indexdef - Get the definition of an index
*
@@ -1891,6 +1888,13 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+/* Internal version that reports the full partition key definition */
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2144,6 +2148,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -3513,7 +3526,7 @@ pg_get_function_arg_default(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(string_to_text(str));
}
-static void
+void
print_function_sqlbody(StringInfo buf, HeapTuple proctup)
{
int numargs;
@@ -12053,7 +12066,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12447,7 +12460,7 @@ generate_qualified_relation_name(Oid relid)
*
* The result includes all necessary quoting and schema-prefixing.
*/
-static char *
+char *
generate_function_name(Oid funcid, int nargs, List *argnames, Oid *argtypes,
bool has_variadic, bool *use_variadic_p,
ParseExprKind special_exprkind)
@@ -12833,7 +12846,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2e41f4d..e38230d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11886,4 +11886,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'deparse the DDL command into json format string',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index d38c0e2..4ca84b2 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..bdfdf47
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.h
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/ddl_deparse.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd, bool verbose_mode);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior);
+
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 221c3e6..85a62a1 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid, int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
extern int32 type_maximum_size(Oid type_oid, int32 typemod);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..3d69a60 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -13,6 +13,7 @@
#ifndef RULEUTILS_H
#define RULEUTILS_H
+#include "catalog/pg_trigger.h"
#include "nodes/nodes.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
@@ -23,12 +24,16 @@ struct PlannedStmt;
extern char *pg_get_indexdef_string(Oid indexrelid);
extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
+extern char *pg_get_trigger_whenclause(Form_pg_trigger trigrec,
+ Node *whenClause, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -40,8 +45,16 @@ extern List *select_rtable_names_for_explain(List *rtable,
Bitmapset *rels_used);
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
+extern char *generate_function_name(Oid funcid, int nargs, List *argnames,
+ Oid *argtypes, bool has_variadic,
+ bool *use_variadic_p,
+ ParseExprKind special_exprkind);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
+extern void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
#endif /* RULEUTILS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 60709dd..a4a6c8f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1621,6 +1621,9 @@ OSInfo
OSSLCipher
OSSLDigest
OVERLAPPED
+ObjElem
+ObjTree
+ObjType
ObjectAccessDrop
ObjectAccessNamespaceSearch
ObjectAccessPostAlter
@@ -3182,6 +3185,7 @@ colormaprange
compare_context
config_var_value
contain_aggs_of_level_context
+convSpecifier
convert_testexpr_context
copy_data_source_cb
core_YYSTYPE
--
2.7.2.windows.1
[application/octet-stream] v14-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (15.1K, ../../OS0PR01MB57168B82B65B491596F5832894969@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v14-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From 73b60e9d5a84615b2d17bbf20ee8ed43e838ee1f Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 28 Jul 2022 10:39:17 +0800
Subject: [PATCH v14 3/4] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 24 +++++
src/backend/commands/event_trigger.c | 172 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 232 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index d6343db..2a03bf5 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -4919,6 +4919,27 @@ deparse_DefineStmt(Oid objectId, Node *parsetree, ObjectAddress secondaryObj)
}
/*
+ * Deparse CREATE TABLE AS command.
+ *
+ * deparse_CreateStmt do the actual work as we deparse the final CreateStmt for
+ * CREATE TABLE AS command.
+ */
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
+/*
* Handle deparsing of simple commands.
*
* This function should cover all cases handled in ProcessUtilitySlow.
@@ -5079,6 +5100,9 @@ deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
case SCT_AlterOpFamily:
tree = deparse_AlterOpFamily(cmd);
break;
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index a00f061..72b83a1 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -585,7 +587,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -868,6 +871,163 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ /* Do nothing if no command was collected. */
+ if (!currentEventTriggerState->currentCommand)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1148,7 +1308,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1867,6 +2028,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1884,6 +2046,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 53bdc77..3a041b2 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -978,6 +978,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -989,6 +994,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6b0a865..490b73b 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index f7f7165..7fb8cb2 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 571b9b6..a4adc9a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11901,4 +11901,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
[application/octet-stream] v14-0002-Support-DDL-replication.patch (123.8K, ../../OS0PR01MB57168B82B65B491596F5832894969@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v14-0002-Support-DDL-replication.patch)
download | inline diff:
From b31243ed1ffbb9e1c35418f857a20a3e9a904f1b Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 28 Jul 2022 10:29:23 +0800
Subject: [PATCH v14 2/4] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
- And other table unrelated command
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 214 ++++++++++++-
src/backend/commands/publicationcmds.c | 112 +++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 48 +++
src/backend/replication/logical/reorderbuffer.c | 136 +++++++++
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
31 files changed, 1591 insertions(+), 213 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index ade3bf3..0a4c01d 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index eef3e5d..a00f061 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1536,6 +1539,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1569,7 +1573,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1589,6 +1593,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2174,3 +2179,210 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged
+ * tables can also be created and altered on the subscriber side. This
+ * makes it possible to directly replay the SET LOGGED command and the
+ * incoming rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist = RELPERSISTENCE_PERMANENT;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ if (get_rel_relkind(relid))
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+ DeparsedCommandType cmdtype;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ cmdtype = DCT_TableDropEnd;
+ else if (strncmp(obj->objecttype, "sequence", 5) == 0 ||
+ strncmp(obj->objecttype, "schema", 5) == 0 ||
+ strncmp(obj->objecttype, "index", 5) == 0)
+ cmdtype = DCT_ObjectDrop;
+ else
+ continue;
+
+ command = deparse_drop_command(obj->objidentity, obj->objecttype,
+ stmt->behavior);
+
+ if (command == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, cmdtype,
+ command, strlen(command) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 89a0055..53bdc77 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,61 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_CREATE_OPERATOR,
+ CMDTAG_CREATE_OPERATOR_CLASS,
+ CMDTAG_CREATE_OPERATOR_FAMILY,
+ CMDTAG_ALTER_OPERATOR_FAMILY,
+ CMDTAG_ALTER_OPERATOR_CLASS,
+ CMDTAG_ALTER_OPERATOR,
+ CMDTAG_CREATE_PROCEDURE,
+ CMDTAG_ALTER_PROCEDURE,
+ CMDTAG_CREATE_FUNCTION,
+ CMDTAG_ALTER_FUNCTION,
+ CMDTAG_CREATE_TRIGGER,
+ CMDTAG_ALTER_TRIGGER,
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1131,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0da73bb..3c058ed 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5223,7 +5223,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index c5c6a2b..e4e0b62 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..f354062 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 88a37fd..1f6d4ea 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4299,6 +4408,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5f8c541..443a3c2 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index a3c1ba8..7bf63dc 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -53,6 +53,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -426,6 +433,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -499,6 +507,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1381,9 +1390,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1417,6 +1439,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1446,8 +1475,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1467,8 +1496,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1598,6 +1638,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1675,8 +1718,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1716,6 +1759,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Return true if the data is associated with an origin and the user has
* requested the changes that don't have an origin, false otherwise.
@@ -1997,7 +2171,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2055,6 +2230,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2168,6 +2344,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2353,6 +2530,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index bdb771d..6c10974 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5627,6 +5627,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f9c51d1..af0319c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3873,6 +3873,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3888,23 +3889,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3920,6 +3927,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3943,6 +3951,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4022,6 +4032,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 69ee939..6c55de3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 327a694..77c4711 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6159,7 +6159,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6194,6 +6194,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6284,6 +6288,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6300,6 +6305,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6313,6 +6319,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6364,6 +6373,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6378,6 +6389,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6388,6 +6401,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e38230d..571b9b6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11892,4 +11892,13 @@
{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index b7d28d7..763e43f 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -91,6 +91,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -202,6 +214,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -221,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -239,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index 0202755..83e0b1e 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index d109d0b..737284c 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index a7f5700..bfc73c2 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5970,9 +5970,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 428c1f1..1df5951 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v14-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB57168B82B65B491596F5832894969@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v14-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From 7e660dd651b7bdbcdb29a5f86e24bfbb378d409c Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH v14 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-08-01 04:18 Peter Smith <smithpb2250@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
1 sibling, 1 reply; 449+ messages in thread
From: Peter Smith @ 2022-08-01 04:18 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
FYI, I found that the v14-0001 patch does not currently apply [1]. Can
you please rebase it?
------
[1] http://cfbot.cputube.org/patch_38_3595.log
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 449+ messages in thread
* RE: Support logical replication of DDLs
@ 2022-08-02 03:20 houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
parent: Peter Smith <smithpb2250@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: houzj.fnst@fujitsu.com @ 2022-08-02 03:20 UTC (permalink / raw)
To: Peter Smith <smithpb2250@gmail.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
On Monday, August 1, 2022 12:18 PM Peter Smith <smithpb2250@gmail.com> wrote:
>
> FYI, I found that the v14-0001 patch does not currently apply [1]. Can you please
> rebase it?
Thanks for reporting. Here is the rebased version.
Best regards,
Hou zj
Attachments:
[application/octet-stream] v15-0004-Test-cases-for-DDL-replication.patch (22.5K, ../../OS0PR01MB5716009FDCCC0B50BCB14A99949D9@OS0PR01MB5716.jpnprd01.prod.outlook.com/2-v15-0004-Test-cases-for-DDL-replication.patch)
download | inline diff:
From ee70f2b56c62115a27f82869d5c63e14303e6e2d Mon Sep 17 00:00:00 2001
From: Takamichi Osumi <osumi.takamichi@fujitsu.com>
Date: Wed, 15 Jun 2022 09:45:42 +0000
Subject: [PATCH v15 4/4] Test cases for DDL replication
Author: Takamichi Osumi
---
src/test/subscription/t/032_ddl_replication.pl | 414 +++++++++++++++++++++++++
1 file changed, 414 insertions(+)
create mode 100644 src/test/subscription/t/032_ddl_replication.pl
diff --git a/src/test/subscription/t/032_ddl_replication.pl b/src/test/subscription/t/032_ddl_replication.pl
new file mode 100644
index 0000000..94de8ed
--- /dev/null
+++ b/src/test/subscription/t/032_ddl_replication.pl
@@ -0,0 +1,414 @@
+# Copyright (c) 2022, PostgreSQL Global Development Group
+# Regression tests for logical replication of DDLs
+#
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+my $node_publisher = PostgreSQL::Test::Cluster->new('publisher');
+$node_publisher->init(allows_streaming => 'logical');
+$node_publisher->start;
+
+my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber');
+$node_subscriber->init(allows_streaming => 'logical');
+$node_subscriber->start;
+
+my $ddl = "CREATE TABLE test_rep(id int primary key, name varchar, value integer);";
+$node_publisher->safe_psql('postgres', $ddl);
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (1, 'data1', 1);");
+$node_subscriber->safe_psql('postgres', $ddl);
+
+my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres';
+$node_publisher->safe_psql('postgres',
+ "CREATE PUBLICATION mypub FOR ALL TABLES with (publish = 'insert, update, delete, ddl');");
+$node_subscriber->safe_psql('postgres',
+ "CREATE SUBSCRIPTION mysub CONNECTION '$publisher_connstr' PUBLICATION mypub;");
+$node_publisher->wait_for_catchup('mysub');
+
+# Make sure we have fully synchronized the table.
+# This prevents ALTER TABLE command below from being executed during table synchronization.
+$node_subscriber->poll_query_until('postgres',
+ "SELECT COUNT(1) = 0 FROM pg_subscription_rel sr WHERE sr.srsubstate NOT IN ('s', 'r') AND sr.srrelid = 'test_rep'::regclass"
+);
+
+# Test ALTER TABLE ADD
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ADD c4 int;");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (2, 'data2', 2, 2);");
+$node_publisher->wait_for_catchup('mysub');
+my $result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE c4 = 2;");
+is($result, qq(1), 'ALTER test_rep ADD replicated');
+
+# Test ALTER TABLE DROP
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep DROP c4;");
+$node_publisher->safe_psql('postgres', "DELETE FROM test_rep where id = 2;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from test_rep;");
+is($result, qq(1), 'ALTER test_rep DROP replicated');
+
+# Test ALTER TABLE ALTER TYPE
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value TYPE varchar");
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (3, 'data3', '3');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE value = '3';");
+is($result, qq(1), 'ALTER test_rep ALTER COLUMN TYPE replicated');
+
+# Test ALTER TABLE ALTER SET DEFAULT
+# Check if we have the default value after the direct insert to subscriber node.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value SET DEFAULT 'foo'");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (4, 'data4');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value FROM test_rep WHERE id = 4;");
+is($result, 'foo', 'ALTER test_rep ALTER SET DEFAULT replicated');
+
+# Test ALTER TABLE ALTER DROP DEFAULT
+# Check if we don't have the default value previously defined.
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER COLUMN value DROP DEFAULT;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (5, 'data5');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 5;");
+is($result, q(t), 'ALTER test_rep ALTER DROP DEFAULT replicated');
+
+# Test ALTER TABLE ALTER SET NOT NULL
+# Remove the existing record that contains null value first.
+my ($stdout, $stderr);
+$node_subscriber->safe_psql('postgres', "DELETE FROM test_rep WHERE id = 5;");
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value SET NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column \"value\" of relation \"test_rep\" violates not-null constraint/
+ or die "failed to replicate ALTER TABLE ALTER SET NOT NULL";
+
+# Test ALTER TABLE ALTER DROP NOT NULL
+$node_publisher->safe_psql('postgres', "ALTER TABLE test_rep ALTER value DROP NOT NULL;");
+$node_publisher->wait_for_catchup('mysub');
+# Insert same data that has NULL value. This failed before but now should succeed.
+$node_subscriber->safe_psql('postgres', "INSERT INTO test_rep VALUES (6, 'data6');");
+$result = $node_subscriber->safe_psql('postgres', "SELECT value IS NULL FROM test_rep WHERE id = 6;");
+is($result, q(t), "ALTER test_rep ALTER DROP NOT NULL replicated");
+
+# Test ALTER TABLE SET UNLOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET UNLOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (7, 'data7', '7');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 7;");
+is($result, qq(0), 'ALTER test_rep SET UNLOGGED replicated');
+
+# Test ALTER TABLE SET LOGGED
+$node_publisher->safe_psql('postgres', 'ALTER TABLE test_rep SET LOGGED;');
+$node_publisher->wait_for_catchup('mysub');
+$node_publisher->safe_psql('postgres', "INSERT INTO test_rep VALUES (8, 'data8', '8');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) FROM test_rep WHERE id = 8;");
+is($result, qq(1), 'ALTER test_rep SET LOGGED replicated');
+
+# Test CREATE TABLE and DML changes
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (a int, b varchar);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE tmp is replicated');
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (1, 'a')");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp values (2, 'b')");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(2), 'DML Changes to tmp are replicated');
+
+# Test CREATE TABLE INHERITS
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp2 (c3 int) INHERITS (tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp2 VALUES (1, 'a', 1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp2';");
+is($result, qq(1), 'CREATE TABLE INHERITS is replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp2;");
+is($result, qq(1), 'inserting some data to inherited table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp2");
+
+# Test CREATE TABLE LIKE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp3 (c3 int, LIKE tmp);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp3 VALUES (1, 1, 'a');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp3';");
+is($result, qq(1), 'CREATE TABLE LIKE replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp3;");
+is($result, qq(1), 'insert some data to a table defined with LIKE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp3");
+
+# Test DROP TABLE
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(0), 'TABLE tmp is dropped');
+
+# Test CREATE UNLOGGED TABLE
+$node_publisher->safe_psql('postgres', "CREATE UNLOGGED TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE UNLOGGED TABLE is replicated correctly');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(0), 'inserting data to unlogged table is not replicated correctly');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from pg_tables where tablename = 'tmp';");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'inserting data to a table replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the subscriber in advance, and if we succeed
+# in replicating changes.)
+$node_subscriber->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE IF NOT EXISTS (check if we skip to create a table
+# when we have the table on the publisher, but not on the subscriber.)
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE IF NOT EXISTS tmp (id int);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE IF NOT EXISTS replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with collate
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (name text COLLATE \"C\");");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES ('foo');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT collation_name FROM information_schema.columns WHERE table_name = 'tmp';");
+is($result, qq(C), 'CREATE TABLE with collate replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# Test CREATE TABLE with named constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int CONSTRAINT \"must be bigger than 10\" CHECK (id > 10));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "must be bigger than 10"/
+ or die "failed to replicate named constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with various types of constraints.
+# NOT NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NOT NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: null value in column "name" of relation "tmp" violates not-null constraint/
+ or die "failed to replicate non null constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NULL constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text NULL);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name IS NULL FROM tmp;");
+is($result, qq(t), "CREATE TABLE with NULL constraint replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# CHECK constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, product_ame text, price int CHECK (price > 0));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'foo', -100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_price_check"/
+ or die "failed to replicate CHECK constraint";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFAULT
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int, name text DEFAULT 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$result = $node_subscriber->safe_psql('postgres', "SELECT name from tmp;");
+is($result, qq(foo), "CREATE TABLE with default value replicated");
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+# UNIQUE constraint
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int UNIQUE);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_id_key"/
+ or die "failed to replicate constraint at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# PRIMARY KEY
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate primary key at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# EXCLUDE
+$node_publisher->safe_psql('postgres', "CREATE TABLE circles (c circle, EXCLUDE USING gist (c WITH &&));");
+$node_publisher->safe_psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO circles VALUES ('<(1, 1), 1>'::circle);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: conflicting key value violates exclusion constraint "circles_c_excl"/
+ or die "failed to replicate EXCLUDE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE circles");
+
+# REFERENCES
+$node_publisher->safe_psql('postgres', "CREATE TABLE product (id int PRIMARY KEY, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO product VALUES (2, 'bar');");
+$node_publisher->safe_psql('postgres', "CREATE TABLE orders (order_id int PRIMARY KEY, product_id int REFERENCES product (id))");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO orders VALUES (1, 10)",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: insert or update on table "orders" violates foreign key constraint "orders_product_id_fkey"/
+ or die "failed to replicate REFERENCES at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE orders");
+$node_publisher->safe_psql('postgres', "DROP TABLE product");
+
+# DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should fail for INITIALLY IMMEDIATE(the default)
+$node_subscriber->psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+), on_error_stop => 0, stderr => \$stderr, stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# NOT DEFERRABLE
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY NOT DEFERRABLE, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "UPDATE tmp SET id = id + 1;",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: duplicate key value violates unique constraint "tmp_pkey"/
+ or die "failed to replicate NOT DEFERRABLE at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# DEFERRABLE and INITIALLY DEFERRED
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int PRIMARY KEY DEFERRABLE INITIALLY DEFERRED, name text);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'foo');");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (2, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+# Quick check of deferrable clause
+$node_subscriber->safe_psql('postgres', "UPDATE tmp SET id = id + 1;");
+# Also, execute a test that should succeed for INITIALLY DEFERRED
+$node_subscriber->safe_psql('postgres', qq(
+BEGIN;
+UPDATE tmp SET id = id + 1;
+INSERT INTO tmp VALUES (3, 'foobar');
+DELETE FROM tmp WHERE id = 3;
+COMMIT;
+));
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE with table constraint
+# We will set two checks and conduct two inserts that should fail respectively.
+$node_publisher->safe_psql('postgres',
+ "CREATE TABLE tmp (price int, discounted_price int, CHECK (discounted_price > 0 AND price > discounted_price));");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (100, 0);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (first condition) at creating table";
+$node_subscriber->psql('postgres', "INSERT INTO tmp VALUES (50, 100);",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stderr =~ /ERROR: new row for relation "tmp" violates check constraint "tmp_check"/
+ or die "failed to replicate table constraint (second condition) at creating table";
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE WITH strorage_parameter
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) WITH (fillfactor = 80);");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$node_subscriber->psql('postgres', "SELECT reloptions FROM pg_class WHERE relname = 'tmp';",
+ on_error_stop => 0,
+ stderr => \$stderr,
+ stdout => \$stdout);
+$stdout =~ /{fillfactor=80}/
+ or die "failed to replicate storage option";
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE with storage_parameter replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE TABLESPACE (creating a tablespace is not replicated)
+# Prepare the directories for the publisher and subscriber first.
+my ($basedir, $tablespace_dir);
+
+$basedir = $node_publisher->basedir;
+$tablespace_dir = "$basedir/tblspc_pub";
+mkdir($tablespace_dir);
+$node_publisher->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+$basedir = $node_subscriber->basedir;
+$tablespace_dir = "$basedir/tblspc_sub";
+mkdir ($tablespace_dir);
+$node_subscriber->safe_psql('postgres', "CREATE TABLESPACE mytblspc LOCATION '$tablespace_dir';");
+
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp (id int) TABLESPACE mytblspc;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1);");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE TABLESPACE replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp;");
+
+# Test CREATE TABLE OF (creating a type is not replicated)
+$node_publisher->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_subscriber->safe_psql('postgres', "CREATE TYPE mytype AS (id int, name text, age int);");
+$node_publisher->safe_psql('postgres', "CREATE TABLE tmp OF mytype;");
+$node_publisher->safe_psql('postgres', "INSERT INTO tmp VALUES (1, 'bar');");
+$node_publisher->wait_for_catchup('mysub');
+$result = $node_subscriber->safe_psql('postgres', "SELECT count(*) from tmp;");
+is($result, qq(1), 'CREATE TABLE OF replicated');
+$node_publisher->safe_psql('postgres', "DROP TABLE tmp");
+
+pass "DDL replication tests passed:";
+
+$node_subscriber->stop;
+$node_publisher->stop;
+
+done_testing();
--
2.7.2.windows.1
[application/octet-stream] v15-0001-Functions-to-deparse-DDL-commands.patch (182.6K, ../../OS0PR01MB5716009FDCCC0B50BCB14A99949D9@OS0PR01MB5716.jpnprd01.prod.outlook.com/3-v15-0001-Functions-to-deparse-DDL-commands.patch)
download | inline diff:
From 728f903d6ae45b375a65d79a07f017ac764822c4 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Tue, 2 Aug 2022 10:08:51 +0800
Subject: [PATCH v15 1/4] Functions to deparse DDL commands.
This patch provides JSON blobs representing DDL commands, which can
later be re-processed into plain strings by well-defined sprintf-like
expansion. These JSON objects are intended to allow for machine-editing of
the commands, by replacing certain nodes within the objects.
Much of the information in the output blob actually comes from system
catalogs, not from the command parse node, as it is impossible to reliably
construct a fully-specified command (i.e. one not dependent on search_path
etc) looking only at the parse node.
This provides a base for logical replication of DDL statements. Currently,
the patch has support for:
CREATE/ALTER/DROP TABLE (*)
CREATE/ALTER/DROP SEQUENCE
CREATE/ALTER/DROP SCHEMA
CREATE/ALTER FUNCTION/PROCEDURE
CREATE/ALTER TRIGGER
CREATE/DROP INDEX
CREATE/DROP OPERATOR [CLASS/FAMILY]
(*) Note that some recently introduced DDLs(e.g. DDLs related to
PARTITIONED TABLE) are unsupported. We can extend it as we need more
functionality for DDL replication.
(*) Note that, for ATTACH/DETACH PARTITION, we haven't added extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
---
src/backend/commands/Makefile | 2 +
src/backend/commands/ddl_deparse.c | 5135 +++++++++++++++++++++++++++++++++++
src/backend/commands/ddl_json.c | 760 ++++++
src/backend/commands/sequence.c | 33 +
src/backend/utils/adt/format_type.c | 3 +-
src/backend/utils/adt/ruleutils.c | 149 +-
src/include/catalog/pg_proc.dat | 6 +
src/include/commands/sequence.h | 1 +
src/include/tcop/ddl_deparse.h | 23 +
src/include/tcop/deparse_utility.h | 1 +
src/include/utils/builtins.h | 1 +
src/include/utils/ruleutils.h | 13 +
src/tools/pgindent/typedefs.list | 4 +
13 files changed, 6061 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/ddl_deparse.c
create mode 100644 src/backend/commands/ddl_json.c
create mode 100644 src/include/tcop/ddl_deparse.h
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 48f7348..171dfb2 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -29,6 +29,8 @@ OBJS = \
copyto.o \
createas.o \
dbcommands.o \
+ ddl_deparse.o \
+ ddl_json.o \
define.o \
discard.o \
dropcmds.o \
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
new file mode 100644
index 0000000..bd109a6
--- /dev/null
+++ b/src/backend/commands/ddl_deparse.c
@@ -0,0 +1,5135 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.c
+ * Functions to convert utility commands to machine-parseable
+ * representation
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * NOTES
+ *
+ * This is intended to provide JSON blobs representing DDL commands, which can
+ * later be re-processed into plain strings by well-defined sprintf-like
+ * expansion. These JSON objects are intended to allow for machine-editing of
+ * the commands, by replacing certain nodes within the objects.
+ *
+ * Much of the information in the output blob actually comes from system
+ * catalogs, not from the command parse node, as it is impossible to reliably
+ * construct a fully-specified command (i.e. one not dependent on search_path
+ * etc) looking only at the parse node.
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_deparse.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "tcop/ddl_deparse.h"
+
+#include "access/amapi.h"
+#include "access/table.h"
+#include "access/relation.h"
+#include "catalog/namespace.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_class.h"
+#include "catalog/pg_collation.h"
+#include "catalog/pg_constraint.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_inherits.h"
+#include "catalog/pg_language.h"
+#include "catalog/pg_namespace.h"
+#include "catalog/pg_opclass.h"
+#include "catalog/pg_operator.h"
+#include "catalog/pg_opfamily.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_sequence.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "commands/sequence.h"
+#include "commands/tablespace.h"
+#include "funcapi.h"
+#include "lib/ilist.h"
+#include "nodes/makefuncs.h"
+#include "nodes/nodeFuncs.h"
+#include "nodes/parsenodes.h"
+#include "optimizer/optimizer.h"
+#include "rewrite/rewriteHandler.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/guc.h"
+#include "utils/jsonb.h"
+#include "utils/lsyscache.h"
+#include "utils/memutils.h"
+#include "utils/rel.h"
+#include "utils/ruleutils.h"
+#include "utils/syscache.h"
+
+/* Estimated length of the generated jsonb string */
+#define JSONB_ESTIMATED_LEN 128
+
+/*
+ * Before they are turned into JSONB representation, each command is
+ * represented as an object tree, using the structs below.
+ */
+typedef enum
+{
+ ObjTypeNull,
+ ObjTypeBool,
+ ObjTypeString,
+ ObjTypeArray,
+ ObjTypeInteger,
+ ObjTypeFloat,
+ ObjTypeObject
+} ObjType;
+
+typedef struct ObjTree
+{
+ slist_head params;
+ int numParams;
+ StringInfo fmtinfo;
+ bool present;
+} ObjTree;
+
+typedef struct ObjElem
+{
+ char *name;
+ ObjType objtype;
+
+ union
+ {
+ bool boolean;
+ char *string;
+ int64 integer;
+ float8 flt;
+ ObjTree *object;
+ List *array;
+ } value;
+ slist_node node;
+} ObjElem;
+
+bool verbose = true;
+
+static ObjElem *new_object(ObjType type, char *name);
+static ObjElem *new_string_object(char *value);
+static ObjElem *new_object_object(ObjTree *value);
+static void append_null_object(ObjTree *tree, char *name);
+static void append_bool_object(ObjTree *tree, char *name, bool value);
+static void append_string_object(ObjTree *tree, char *name, char *value);
+static void append_object_object(ObjTree *tree, char *name, ObjTree *value);
+static void append_array_object(ObjTree *tree, char *name, List *array);
+static void append_float_object(ObjTree *tree, char *name, float8 value);
+static inline void append_premade_object(ObjTree *tree, ObjElem *elem);
+static JsonbValue *objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state);
+static void format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray);
+static const char *stringify_objtype(ObjectType objtype);
+
+static char *RelationGetColumnDefault(Relation rel, AttrNumber attno,
+ List *dpcontext, List **exprs);
+static List *FunctionGetDefaults(text *proargdefaults);
+
+static inline ObjElem *deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table);
+static inline ObjElem *deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata);
+
+/*
+ * Similar to format_type_extended, except we return each bit of information
+ * separately:
+ *
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
+ *
+ * - typename is set to the type name, without quotes
+ *
+ * - typmod is set to the typemod, if any, as a string with parens
+ *
+ * - typarray indicates whether []s must be added
+ *
+ * We don't try to decode type names to their standard-mandated names, except
+ * in the cases of types with unusual typmod rules.
+ */
+static void
+format_type_detailed(Oid type_oid, int32 typemod,
+ Oid *nspid, char **typname, char **typemodstr,
+ bool *typarray)
+{
+ HeapTuple tuple;
+ Form_pg_type typeform;
+ Oid array_base_type;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(type_oid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+
+ /*
+ * Check if it's a regular (variable length) array type. As above,
+ * fixed-length array types such as "name" shouldn't get deconstructed.
+ */
+ array_base_type = typeform->typelem;
+
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
+
+ /*
+ * Special-case crock for types with strange typmod rules where we put
+ * typmod at the middle of name(e.g. TIME(6) with time zone ). We cannot
+ * schema-qualify nor add quotes to the type name in these cases.
+ */
+ if (type_oid == INTERVALOID ||
+ type_oid == TIMESTAMPOID ||
+ type_oid == TIMESTAMPTZOID ||
+ type_oid == TIMEOID ||
+ type_oid == TIMETZOID)
+ {
+ switch (type_oid)
+ {
+ case INTERVALOID:
+ *typname = pstrdup("INTERVAL");
+ break;
+ case TIMESTAMPTZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIMESTAMP WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmod. */
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMESTAMPOID:
+ *typname = pstrdup("TIMESTAMP");
+ break;
+ case TIMETZOID:
+ if (typemod < 0)
+ *typname = pstrdup("TIME WITH TIME ZONE");
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
+ break;
+ case TIMEOID:
+ *typname = pstrdup("TIME");
+ break;
+ }
+ *nspid = InvalidOid;
+ }
+ else
+ {
+ /*
+ * No additional processing is required for other types, so get the type
+ * name and schema directly from the catalog.
+ */
+ *nspid = typeform->typnamespace;
+ *typname = pstrdup(NameStr(typeform->typname));
+ }
+
+ if (typemod >= 0)
+ *typemodstr = printTypmod("", typemod, typeform->typmodout);
+ else
+ *typemodstr = pstrdup("");
+
+ ReleaseSysCache(tuple);
+}
+
+/*
+ * Obtain the deparsed default value for the given column of the given table.
+ *
+ * Caller must have set a correct deparse context.
+ */
+static char *
+RelationGetColumnDefault(Relation rel, AttrNumber attno, List *dpcontext,
+ List **exprs)
+{
+ Node *defval;
+ char *defstr;
+
+ defval = build_column_default(rel, attno);
+ defstr = deparse_expression(defval, dpcontext, false, false);
+
+ /* Collect the expression for later replication safety checks */
+ if (exprs)
+ *exprs = lappend(*exprs, defval);
+
+ return defstr;
+}
+
+/*
+ * Return the defaults values of arguments to a function, as a list of
+ * deparsed expressions.
+ */
+static List *
+FunctionGetDefaults(text *proargdefaults)
+{
+ List *nodedefs;
+ List *strdefs = NIL;
+ ListCell *cell;
+
+ nodedefs = (List *) stringToNode(TextDatumGetCString(proargdefaults));
+ if (!IsA(nodedefs, List))
+ elog(ERROR, "proargdefaults is not a list");
+
+ foreach(cell, nodedefs)
+ {
+ Node *onedef = lfirst(cell);
+
+ strdefs = lappend(strdefs, deparse_expression(onedef, NIL, false, false));
+ }
+
+ return strdefs;
+}
+
+/*
+ * Allocate a new object tree to store parameter values.
+ */
+static ObjTree *
+new_objtree(char *fmt)
+{
+ ObjTree *params;
+
+ params = palloc0(sizeof(ObjTree));
+ params->present = true;
+ slist_init(¶ms->params);
+
+ if (fmt)
+ {
+ params->fmtinfo = makeStringInfo();
+ appendStringInfoString(params->fmtinfo, fmt);
+ }
+
+ return params;
+}
+
+/*
+ * Allocate a new object tree to store parameter values -- varargs version.
+ *
+ * The "fmt" argument is used to append as a "fmt" element in the output blob.
+ * numobjs indicates the number of extra elements to append; for each one, a
+ * name (string), type (from the ObjType enum) and value must be supplied. The
+ * value must match the type given; for instance, ObjTypeInteger requires an
+ * int64, ObjTypeString requires a char *, ObjTypeArray requires a list (of
+ * ObjElem), ObjTypeObject requires an ObjTree, and so on. Each element type *
+ * must match the conversion specifier given in the format string, as described
+ * in ddl_deparse_expand_command, q.v.
+ *
+ * Note we don't have the luxury of sprintf-like compiler warnings for
+ * malformed argument lists.
+ */
+static ObjTree *
+new_objtree_VA(char *fmt, int numobjs,...)
+{
+ ObjTree *tree;
+ va_list args;
+ int i;
+
+ /* Set up the toplevel object and its "fmt" */
+ tree = new_objtree(fmt);
+
+ /* And process the given varargs */
+ va_start(args, numobjs);
+ for (i = 0; i < numobjs; i++)
+ {
+ char *name;
+ ObjType type;
+ ObjElem *elem;
+
+ name = va_arg(args, char *);
+ type = va_arg(args, ObjType);
+ elem = new_object(type, NULL);
+
+ /*
+ * For all other param types there must be a value in the varargs.
+ * Fetch it and add the fully formed subobject into the main object.
+ */
+ switch (type)
+ {
+ case ObjTypeNull:
+ /* Null params don't have a value (obviously) */
+ break;
+ case ObjTypeBool:
+ elem->value.boolean = va_arg(args, int);
+ break;
+ case ObjTypeString:
+ elem->value.string = va_arg(args, char *);
+ break;
+ case ObjTypeArray:
+ elem->value.array = va_arg(args, List *);
+ break;
+ case ObjTypeInteger:
+ elem->value.integer = va_arg(args, int64);
+ break;
+ case ObjTypeFloat:
+ elem->value.flt = va_arg(args, double);
+ break;
+ case ObjTypeObject:
+ elem->value.object = va_arg(args, ObjTree *);
+ break;
+ default:
+ elog(ERROR, "invalid ObjTree element type %d", type);
+ }
+
+ elem->name = name;
+ append_premade_object(tree, elem);
+ }
+
+ va_end(args);
+ return tree;
+}
+
+/*
+ * Append the input format string to the ObjTree.
+ */
+static void
+append_format_string(ObjTree *tree, char *sub_fmt)
+{
+ int len;
+ char *fmt;
+
+ if (!tree->fmtinfo)
+ return;
+
+ fmt = tree->fmtinfo->data;
+ len = tree->fmtinfo->len;
+
+ /* Add a separator if necessary */
+ if (len > 0 && fmt[len - 1] != ' ')
+ appendStringInfoSpaces(tree->fmtinfo, 1);
+
+ appendStringInfoString(tree->fmtinfo, sub_fmt);
+}
+
+/*
+ * Return the object name which is extracted from the input "*%{name[:.]}*"
+ * style string. And append the input format string to the ObjTree.
+ */
+static char *
+append_object_to_format_string(ObjTree *tree, char *sub_fmt)
+{
+ StringInfoData object_name;
+ const char *end_ptr;
+ const char *cp;
+ bool start_copy = false;
+
+ if (sub_fmt == NULL || tree->fmtinfo == NULL)
+ return sub_fmt;
+
+ initStringInfo(&object_name);
+ end_ptr = sub_fmt + strlen(sub_fmt);
+
+ for (cp = sub_fmt; cp < end_ptr; cp++)
+ {
+ if (*cp == '{')
+ {
+ start_copy = true;
+ continue;
+ }
+
+ if (!start_copy)
+ continue;
+
+ if (*cp == ':' || *cp == '}')
+ break;
+
+ appendStringInfoCharMacro(&object_name, *cp);
+ }
+
+ if (object_name.len == 0)
+ elog(ERROR, "object name not found");
+
+ append_format_string(tree, sub_fmt);
+
+ return object_name.data;
+}
+
+/* Allocate a new parameter */
+static ObjElem *
+new_object(ObjType type, char *name)
+{
+ ObjElem *param;
+
+ param = palloc0(sizeof(ObjElem));
+ param->name = name;
+ param->objtype = type;
+
+ return param;
+}
+
+/* Append a NULL object to a tree */
+static void
+append_null_object(ObjTree *tree, char *sub_fmt)
+{
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ append_premade_object(tree, new_object(ObjTypeNull, object_name));
+}
+
+/* Append a boolean parameter to a tree */
+static void
+append_bool_object(ObjTree *tree, char *sub_fmt, bool value)
+{
+ ObjElem *param;
+ char *object_name = sub_fmt;
+ bool is_present_flag = false;
+
+ Assert(sub_fmt);
+
+ if (strcmp(sub_fmt, "present") == 0)
+ {
+ is_present_flag = true;
+ tree->present = value;
+ }
+
+ if (!verbose && !tree->present)
+ return;
+
+ if (!is_present_flag)
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeBool, object_name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new string object */
+static ObjElem *
+new_string_object(char *value)
+{
+ ObjElem *param;
+
+ Assert(value);
+
+ param = new_object(ObjTypeString, NULL);
+ param->value.string = value;
+
+ return param;
+}
+
+/* Append a string parameter to a tree */
+static void
+append_string_object(ObjTree *tree, char *sub_fmt, char *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && (value == NULL || value[0] == '\0'))
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeString, object_name);
+ param->value.string = value;
+ append_premade_object(tree, param);
+}
+
+/* Allocate a new object parameter */
+static ObjElem *
+new_object_object(ObjTree *value)
+{
+ ObjElem *param;
+
+ param = new_object(ObjTypeObject, NULL);
+ param->value.object = value;
+
+ return param;
+}
+
+/* Append an object parameter to a tree */
+static void
+append_object_object(ObjTree *tree, char *sub_fmt, ObjTree *value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose && !value->present)
+ return;
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeObject, object_name);
+ param->value.object = value;
+ append_premade_object(tree, param);
+}
+
+/* Append an array parameter to a tree */
+static void
+append_array_object(ObjTree *tree, char *sub_fmt, List *array)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ if (!verbose)
+ {
+ ListCell *lc;
+
+ /* Extract the ObjElems whose present flag is true */
+ foreach(lc, array)
+ {
+ ObjElem *elem = (ObjElem *) lfirst(lc);
+
+ Assert(elem->objtype == ObjTypeObject);
+
+ if (!elem->value.object->present)
+ array = foreach_delete_current(array, lc);
+ }
+
+ if (list_length(array) == 0)
+ return;
+ }
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeArray, object_name);
+ param->value.array = array;
+ append_premade_object(tree, param);
+}
+
+/*
+ * Append a float8 parameter to a tree.
+ */
+static void
+append_float_object(ObjTree *tree, char *sub_fmt, float8 value)
+{
+ ObjElem *param;
+ char *object_name;
+
+ Assert(sub_fmt);
+
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeFloat, object_name);
+ param->value.flt = value;
+ append_premade_object(tree, param);
+}
+
+/* Append a preallocated parameter to a tree */
+static inline void
+append_premade_object(ObjTree *tree, ObjElem *elem)
+{
+ slist_push_head(&tree->params, &elem->node);
+ tree->numParams++;
+}
+
+/*
+ * Process the pre-built format string from the ObjTree into the output parse
+ * state.
+ */
+static void
+objtree_fmt_to_jsonb_element(JsonbParseState *state, ObjTree *tree)
+{
+ JsonbValue key;
+ JsonbValue val;
+
+ if (!tree->fmtinfo)
+ return;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.val = "fmt";
+ key.val.string.len = strlen(key.val.string.val);
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the pre-built format string */
+ val.type = jbvString;
+ val.val.string.len = tree->fmtinfo->len;
+ val.val.string.val = tree->fmtinfo->data;
+ pushJsonbValue(&state, WJB_VALUE, &val);
+}
+
+/*
+ * Helper for objtree_to_jsonb: process an individual element from an object or
+ * an array into the output parse state.
+ */
+static void
+objtree_to_jsonb_element(JsonbParseState *state, ObjElem *object,
+ JsonbIteratorToken elem_token)
+{
+ ListCell *cell;
+ JsonbValue val;
+
+ switch (object->objtype)
+ {
+ case ObjTypeNull:
+ val.type = jbvNull;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeString:
+ val.type = jbvString;
+ val.val.string.len = strlen(object->value.string);
+ val.val.string.val = object->value.string;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeInteger:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(int8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeFloat:
+ val.type = jbvNumeric;
+ val.val.numeric = (Numeric)
+ DatumGetNumeric(DirectFunctionCall1(float8_numeric,
+ object->value.integer));
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeBool:
+ val.type = jbvBool;
+ val.val.boolean = object->value.boolean;
+ pushJsonbValue(&state, elem_token, &val);
+ break;
+
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
+ objtree_to_jsonb_rec(object->value.object, state);
+ break;
+
+ case ObjTypeArray:
+ pushJsonbValue(&state, WJB_BEGIN_ARRAY, NULL);
+ foreach(cell, object->value.array)
+ {
+ ObjElem *elem = lfirst(cell);
+
+ objtree_to_jsonb_element(state, elem, WJB_ELEM);
+ }
+ pushJsonbValue(&state, WJB_END_ARRAY, NULL);
+ break;
+
+ default:
+ elog(ERROR, "unrecognized object type %d", object->objtype);
+ break;
+ }
+}
+
+/*
+ * Recursive helper for objtree_to_jsonb.
+ */
+static JsonbValue *
+objtree_to_jsonb_rec(ObjTree *tree, JsonbParseState *state)
+{
+ slist_iter iter;
+
+ pushJsonbValue(&state, WJB_BEGIN_OBJECT, NULL);
+
+ objtree_fmt_to_jsonb_element(state, tree);
+
+ slist_foreach(iter, &tree->params)
+ {
+ ObjElem *object = slist_container(ObjElem, node, iter.cur);
+ JsonbValue key;
+
+ /* Push the key first */
+ key.type = jbvString;
+ key.val.string.len = strlen(object->name);
+ key.val.string.val = object->name;
+ pushJsonbValue(&state, WJB_KEY, &key);
+
+ /* Then process the value according to its type */
+ objtree_to_jsonb_element(state, object, WJB_VALUE);
+ }
+
+ return pushJsonbValue(&state, WJB_END_OBJECT, NULL);
+}
+
+/*
+ * Create a JSONB representation from an ObjTree.
+ */
+static Jsonb *
+objtree_to_jsonb(ObjTree *tree)
+{
+ JsonbValue *value;
+
+ value = objtree_to_jsonb_rec(tree, NULL);
+ return JsonbValueToJsonb(value);
+}
+
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
+{
+ ObjTree *typeParam;
+ Oid typnspid;
+ char *typnsp;
+ char *typename = NULL;
+ char *typmodstr;
+ bool typarray;
+
+ format_type_detailed(typeId, typmod,
+ &typnspid, &typename, &typmodstr, &typarray);
+
+ if (OidIsValid(typnspid))
+ typnsp = get_namespace_name_or_temp(typnspid);
+ else
+ typnsp = pstrdup("");
+
+ typeParam = new_objtree_VA(NULL, 4,
+ "schemaname", ObjTypeString, typnsp,
+ "typename", ObjTypeString, typename,
+ "typmod", ObjTypeString, typmodstr,
+ "typarray", ObjTypeBool, typarray);
+
+ return typeParam;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements.
+ *
+ * Elements "schemaname" and "objname" are set. If the namespace OID
+ * corresponds to a temp schema, that's set to "pg_temp".
+ *
+ * The difference between those two element types is whether the objname will
+ * be quoted as an identifier or not, which is not something that this routine
+ * concerns itself with; that will be up to the expand function.
+ */
+static ObjTree *
+new_objtree_for_qualname(Oid nspid, char *name)
+{
+ ObjTree *qualified;
+ char *namespace;
+
+ if (isAnyTempNamespace(nspid))
+ namespace = pstrdup("pg_temp");
+ else
+ namespace = get_namespace_name(nspid);
+
+ qualified = new_objtree_VA(NULL, 2,
+ "schemaname", ObjTypeString, namespace,
+ "objname", ObjTypeString, pstrdup(name));
+
+ return qualified;
+}
+
+/*
+ * A helper routine to setup %{}D and %{}O elements, with the object specified
+ * by classId/objId.
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
+ */
+static ObjTree *
+new_objtree_for_qualname_id(Oid classId, Oid objectId)
+{
+ ObjTree *qualified;
+ Relation catalog;
+ HeapTuple catobj;
+ Datum objnsp;
+ Datum objname;
+ AttrNumber Anum_name;
+ AttrNumber Anum_namespace;
+ AttrNumber Anum_oid = get_object_attnum_oid(classId);
+ bool isnull;
+
+ catalog = table_open(classId, AccessShareLock);
+
+ catobj = get_catalog_object_by_oid(catalog, Anum_oid, objectId);
+ if (!catobj)
+ elog(ERROR, "cache lookup failed for object %u of catalog \"%s\"",
+ objectId, RelationGetRelationName(catalog));
+ Anum_name = get_object_attnum_name(classId);
+ Anum_namespace = get_object_attnum_namespace(classId);
+
+ objnsp = heap_getattr(catobj, Anum_namespace, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
+ if (isnull)
+ elog(ERROR, "unexpected NULL name");
+
+ qualified = new_objtree_for_qualname(DatumGetObjectId(objnsp),
+ NameStr(*DatumGetName(objname)));
+ table_close(catalog, AccessShareLock);
+
+ return qualified;
+}
+
+/*
+ * Return the string representation of the given RELPERSISTENCE value.
+ */
+static char *
+get_persistence_str(char persistence)
+{
+ switch (persistence)
+ {
+ case RELPERSISTENCE_TEMP:
+ return "TEMPORARY";
+ case RELPERSISTENCE_UNLOGGED:
+ return "UNLOGGED";
+ case RELPERSISTENCE_PERMANENT:
+ return "";
+ default:
+ elog(ERROR, "unexpected persistence marking %c", persistence);
+ return ""; /* make compiler happy */
+ }
+}
+
+/*
+ * Deparse the definition of column identity.
+ */
+static ObjTree *
+deparse_ColumnIdentity(Oid seqrelid, char identity, bool alter_table)
+{
+ ObjTree *column;
+ ObjTree *identobj;
+ List *elems = NIL;
+ Relation rel;
+ HeapTuple seqtuple;
+ Form_pg_sequence seqform;
+ Form_pg_sequence_data seqdata;
+ char *identfmt;
+ char *objfmt;
+
+ column = new_objtree("");
+
+ if (!OidIsValid(seqrelid))
+ {
+ append_bool_object(column, "present", false);
+ return column;
+ }
+
+ if (alter_table)
+ {
+ identfmt = "SET GENERATED ";
+ objfmt = "%{option}s";
+ }
+ else
+ {
+ identfmt = "GENERATED ";
+ objfmt = "%{option}s AS IDENTITY";
+ }
+
+ identobj = new_objtree(identfmt);
+
+ if (identity == ATTRIBUTE_IDENTITY_ALWAYS)
+ append_string_object(identobj, objfmt, "ALWAYS");
+ else if (identity == ATTRIBUTE_IDENTITY_BY_DEFAULT)
+ append_string_object(identobj, objfmt, "BY DEFAULT");
+ else
+ append_bool_object(identobj, "present", false);
+
+ append_object_object(column, "%{identity_type}s", identobj);
+
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID, seqrelid);
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ seqrelid);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+ seqdata = get_sequence_values(seqrelid);
+
+ /* Definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Cycle(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_IncrementBy(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Minvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Maxvalue(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Startwith(NULL, seqform, alter_table));
+ elems = lappend(elems, deparse_Seq_Restart(NULL, seqdata));
+ /* We purposefully do not emit OWNED BY here */
+
+ if (alter_table)
+ append_array_object(column, "%{seq_definition: }s", elems);
+ else
+ append_array_object(column, "( %{seq_definition: }s )", elems);
+
+ table_close(rel, RowExclusiveLock);
+
+ return column;
+}
+
+/*
+ * Deparse a CreateTrigStmt (CREATE TRIGGER)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateTrigStmt(Oid objectId, Node *parsetree)
+{
+ CreateTrigStmt *node = (CreateTrigStmt *) parsetree;
+ Relation pg_trigger;
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ ObjTree *trigger;
+ ObjTree *tmp;
+ int tgnargs;
+ List *list;
+ List *events;
+
+ pg_trigger = table_open(TriggerRelationId, AccessShareLock);
+
+ trigTup = get_catalog_object_by_oid(pg_trigger, Anum_pg_trigger_oid, objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{constraint}s TRIGGER %{name}I %{time}s %{events: OR }s
+ * ON %{relation}D %{from_table}s %{constraint_attrs: }s
+ * FOR EACH %{for_each}s %{when}s EXECUTE PROCEDURE %{function}s
+ */
+ trigger = new_objtree("CREATE");
+
+ append_string_object(trigger, "%{constraint}s",
+ node->isconstraint ? "CONSTRAINT" : "");
+
+ append_string_object(trigger, "TRIGGER %{name}I", node->trigname);
+
+ if (node->timing == TRIGGER_TYPE_BEFORE)
+ append_string_object(trigger, "%{time}s", "BEFORE");
+ else if (node->timing == TRIGGER_TYPE_AFTER)
+ append_string_object(trigger, "%{time}s", "AFTER");
+ else if (node->timing == TRIGGER_TYPE_INSTEAD)
+ append_string_object(trigger, "%{time}s", "INSTEAD OF");
+ else
+ elog(ERROR, "unrecognized trigger timing value %d", node->timing);
+
+ /*
+ * Decode the events that the trigger fires for. The output is a list;
+ * in most cases it will just be a string with the even name, but when
+ * there's an UPDATE with a list of columns, we return a JSON object.
+ */
+ events = NIL;
+ if (node->events & TRIGGER_TYPE_INSERT)
+ events = lappend(events, new_string_object("INSERT"));
+ if (node->events & TRIGGER_TYPE_DELETE)
+ events = lappend(events, new_string_object("DELETE"));
+ if (node->events & TRIGGER_TYPE_TRUNCATE)
+ events = lappend(events, new_string_object("TRUNCATE"));
+ if (node->events & TRIGGER_TYPE_UPDATE)
+ {
+ if (node->columns == NIL)
+ {
+ events = lappend(events, new_string_object("UPDATE"));
+ }
+ else
+ {
+ ObjTree *update;
+ ListCell *cell;
+ List *cols = NIL;
+
+ /*
+ * Currently only UPDATE OF can be objects in the output JSON, but
+ * we add a "kind" element so that user code can distinguish
+ * possible future new event types.
+ */
+ update = new_objtree_VA("UPDATE OF", 1,
+ "kind", ObjTypeString, "update_of");
+
+ foreach(cell, node->columns)
+ {
+ char *colname = strVal(lfirst(cell));
+
+ cols = lappend(cols,
+ new_string_object(colname));
+ }
+
+ append_array_object(update, "%{columns:, }I", cols);
+
+ events = lappend(events,
+ new_object_object(update));
+ }
+ }
+ append_array_object(trigger, "%{events: OR }s", events);
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid);
+ append_object_object(trigger, "ON %{relation}D", tmp);
+
+ tmp = new_objtree_VA("FROM", 0);
+ if (trigForm->tgconstrrelid)
+ {
+ ObjTree *rel;
+
+ rel = new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgconstrrelid);
+ append_object_object(tmp, "%{relation}D", rel);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{from_table}s", tmp);
+
+ list = NIL;
+ if (node->deferrable)
+ list = lappend(list,
+ new_string_object("DEFERRABLE"));
+ if (node->initdeferred)
+ list = lappend(list,
+ new_string_object("INITIALLY DEFERRED"));
+ append_array_object(trigger, "%{constraint_attrs: }s", list);
+
+ append_string_object(trigger, "FOR EACH %{for_each}s",
+ node->row ? "ROW" : "STATEMENT");
+
+ tmp = new_objtree_VA("WHEN", 0);
+ if (node->whenClause)
+ {
+ Node *whenClause;
+ Datum value;
+ bool isnull;
+
+ value = fastgetattr(trigTup, Anum_pg_trigger_tgqual,
+ RelationGetDescr(pg_trigger), &isnull);
+ if (isnull)
+ elog(ERROR, "bogus NULL tgqual");
+
+ whenClause = stringToNode(TextDatumGetCString(value));
+ append_string_object(tmp, "(%{clause}s)",
+ pg_get_trigger_whenclause(trigForm,
+ whenClause,
+ false));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(trigger, "%{when}s", tmp);
+
+ tmp = new_objtree_VA("%{funcname}D", 1,
+ "funcname", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ trigForm->tgfoid));
+ list = NIL;
+ tgnargs = trigForm->tgnargs;
+ if (tgnargs > 0)
+ {
+ bytea *tgargs;
+ char *argstr;
+ bool isnull;
+ int findx;
+ int lentgargs;
+ char *p;
+
+ tgargs = DatumGetByteaP(fastgetattr(trigTup,
+ Anum_pg_trigger_tgargs,
+ RelationGetDescr(pg_trigger),
+ &isnull));
+ if (isnull)
+ elog(ERROR, "invalid NULL tgargs");
+ argstr = (char *) VARDATA(tgargs);
+ lentgargs = VARSIZE_ANY_EXHDR(tgargs);
+
+ p = argstr;
+ for (findx = 0; findx < tgnargs; findx++)
+ {
+ size_t tlen;
+
+ /* verify that the argument encoding is correct */
+ tlen = strlen(p);
+ if (p + tlen >= argstr + lentgargs)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid argument string (%s) for trigger \"%s\"",
+ argstr, NameStr(trigForm->tgname))));
+
+ list = lappend(list, new_string_object(p));
+
+ p += tlen + 1;
+ }
+ }
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{args:, }L", list); /* might be NIL */
+ append_format_string(tmp, ")");
+
+ append_object_object(trigger, "EXECUTE PROCEDURE %{function}s", tmp);
+
+ table_close(pg_trigger, AccessShareLock);
+
+ return trigger;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
+ *
+ * NOT NULL constraints in the column definition are emitted directly in the
+ * column definition by this routine; other constraints must be emitted
+ * elsewhere (the info in the parse node is incomplete anyway.).
+ */
+static ObjTree *
+deparse_ColumnDef(Relation relation, List *dpcontext, bool composite,
+ ColumnDef *coldef, bool is_alter, List **exprs)
+{
+ ObjTree *column;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ /*
+ * Inherited columns without local definitions must not be emitted.
+ *
+ * XXX maybe it is useful to have them with "present = false" or some such?
+ */
+ if (!coldef->is_local)
+ return NULL;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I %{coltype}T %{compression}s %{default}s %{not_null}s
+ * %{collation}s
+ */
+ column = new_objtree_VA("%{name}I %{coltype}T", 3,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname,
+ "coltype", ObjTypeObject,
+ new_objtree_for_type(typid, typmod));
+
+ /* USING clause */
+ tmp = new_objtree("COMPRESSION");
+ if (coldef->compression)
+ append_string_object(tmp, "%{compression_method}I", coldef->compression);
+ else
+ {
+ append_null_object(tmp, "%{compression_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(column, "%{compression}s", tmp);
+
+ tmp = new_objtree("COLLATE");
+ if (OidIsValid(typcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ typcollation);
+ append_object_object(tmp, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{collation}s", tmp);
+
+ if (!composite)
+ {
+ Oid seqrelid = InvalidOid;
+
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
+ if (is_alter && coldef->is_not_null)
+ saw_notnull = true;
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /* IDENTITY COLUMN */
+ if (coldef->identity)
+ {
+ Oid attno = get_attnum(relid, coldef->colname);
+ seqrelid = getIdentitySequence(relid, attno, false);
+ }
+
+ tmp = deparse_ColumnIdentity(seqrelid, coldef->identity, is_alter);
+ append_object_object(column, "%{identity_column}s", tmp);
+
+ /* GENERATED COLUMN EXPRESSION */
+ tmp = new_objtree("GENERATED ALWAYS AS");
+ if (coldef->generated == ATTRIBUTE_GENERATED_STORED)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, exprs);
+ append_string_object(tmp, "%{generation_expr}s STORED", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(column, "%{generated_column}s", tmp);
+ }
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE OF deparsing.
+ *
+ * Deparse a ColumnDef node within a typed table creation. This is simpler
+ * than the regular case, because we don't have to emit the type declaration,
+ * collation, or default. Here we only return something if the column is being
+ * declared NOT NULL.
+ *
+ * As in deparse_ColumnDef, any other constraint is processed elsewhere.
+ */
+static ObjTree *
+deparse_ColumnDef_typed(Relation relation, List *dpcontext, ColumnDef *coldef)
+{
+ ObjTree *column = NULL;
+ ObjTree *tmp;
+ Oid relid = RelationGetRelid(relation);
+ HeapTuple attrTup;
+ Form_pg_attribute attrForm;
+ Oid typid;
+ int32 typmod;
+ Oid typcollation;
+ bool saw_notnull;
+ ListCell *cell;
+
+ attrTup = SearchSysCacheAttName(relid, coldef->colname);
+ if (!HeapTupleIsValid(attrTup))
+ elog(ERROR, "could not find cache entry for column \"%s\" of relation %u",
+ coldef->colname, relid);
+ attrForm = (Form_pg_attribute) GETSTRUCT(attrTup);
+
+ get_atttypetypmodcoll(relid, attrForm->attnum,
+ &typid, &typmod, &typcollation);
+
+ /*
+ * Search for a NOT NULL declaration. As in deparse_ColumnDef, we rely on
+ * finding a constraint on the column rather than coldef->is_not_null.
+ * (This routine is never used for ALTER cases.)
+ */
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ {
+ saw_notnull = true;
+ break;
+ }
+ }
+
+ if (!saw_notnull && !attrForm->atthasdef)
+ {
+ ReleaseSysCache(attrTup);
+ return NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * %{name}I WITH OPTIONS %{default}s %{not_null}s.
+ */
+ column = new_objtree_VA("%{name}I WITH OPTIONS", 2,
+ "type", ObjTypeString, "column",
+ "name", ObjTypeString, coldef->colname);
+
+ append_string_object(column, "%{not_null}s",
+ saw_notnull ? "NOT NULL" : "");
+
+ tmp = new_objtree("DEFAULT");
+ if (attrForm->atthasdef)
+ {
+ char *defstr;
+
+ defstr = RelationGetColumnDefault(relation, attrForm->attnum,
+ dpcontext, NULL);
+
+ append_string_object(tmp, "%{default}s", defstr);
+ }
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(column, "%{default}s", tmp);
+
+ /*
+ * Generated columns are not supported on typed tables, so we are done.
+ */
+
+ ReleaseSysCache(attrTup);
+
+ return column;
+}
+
+/*
+ * Subroutine for CREATE TABLE deparsing.
+ *
+ * Deal with all the table elements (columns and constraints).
+ *
+ * Note we ignore constraints in the parse node here; they are extracted from
+ * system catalogs instead.
+ */
+static List *
+deparse_TableElements(Relation relation, List *tableElements, List *dpcontext,
+ bool typed, bool composite)
+{
+ List *elements = NIL;
+ ListCell *lc;
+
+ foreach(lc, tableElements)
+ {
+ Node *elt = (Node *) lfirst(lc);
+
+ switch (nodeTag(elt))
+ {
+ case T_ColumnDef:
+ {
+ ObjTree *tree;
+
+ tree = typed ?
+ deparse_ColumnDef_typed(relation, dpcontext,
+ (ColumnDef *) elt) :
+ deparse_ColumnDef(relation, dpcontext,
+ composite, (ColumnDef *) elt,
+ false, NULL);
+ if (tree != NULL)
+ {
+ ObjElem *column;
+
+ column = new_object_object(tree);
+ elements = lappend(elements, column);
+ }
+ }
+ break;
+ case T_Constraint:
+ break;
+ default:
+ elog(ERROR, "invalid node type %d", nodeTag(elt));
+ }
+ }
+
+ return elements;
+}
+
+/*
+ * Subroutine for CREATE TABLE/CREATE DOMAIN deparsing.
+ *
+ * Given a table OID or domain OID, obtain its constraints and append them to
+ * the given elements list. The updated list is returned.
+ *
+ * This works for typed tables, regular tables, and domains.
+ *
+ * Note that CONSTRAINT_FOREIGN constraints are always ignored.
+ */
+static List *
+obtainConstraints(List *elements, Oid relationId, Oid domainId)
+{
+ Relation conRel;
+ ScanKeyData key;
+ SysScanDesc scan;
+ HeapTuple tuple;
+ ObjTree *constr;
+
+ /* only one may be valid */
+ Assert(OidIsValid(relationId) ^ OidIsValid(domainId));
+
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
+ conRel = table_open(ConstraintRelationId, AccessShareLock);
+ if (OidIsValid(relationId))
+ {
+ ScanKeyInit(&key,
+ Anum_pg_constraint_conrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relationId));
+ scan = systable_beginscan(conRel, ConstraintRelidTypidNameIndexId,
+ true, NULL, 1, &key);
+ }
+ else
+ {
+ Assert(OidIsValid(domainId));
+ ScanKeyInit(&key,
+ Anum_pg_constraint_contypid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(domainId));
+ scan = systable_beginscan(conRel, ConstraintTypidIndexId,
+ true, NULL, 1, &key);
+ }
+
+ /*
+ * For each constraint, add a node to the list of table elements. In
+ * these nodes we include not only the printable information ("fmt"), but
+ * also separate attributes to indicate the type of constraint, for
+ * automatic processing.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_constraint constrForm;
+ char *contype;
+
+ constrForm = (Form_pg_constraint) GETSTRUCT(tuple);
+
+ switch (constrForm->contype)
+ {
+ case CONSTRAINT_CHECK:
+ contype = "check";
+ break;
+ case CONSTRAINT_FOREIGN:
+ continue; /* not here */
+ case CONSTRAINT_PRIMARY:
+ contype = "primary key";
+ break;
+ case CONSTRAINT_UNIQUE:
+ contype = "unique";
+ break;
+ case CONSTRAINT_TRIGGER:
+ contype = "trigger";
+ break;
+ case CONSTRAINT_EXCLUSION:
+ contype = "exclusion";
+ break;
+ default:
+ elog(ERROR, "unrecognized constraint type");
+ }
+
+ /*
+ * "type" and "contype" are not part of the printable output, but are
+ * useful to programmatically distinguish these from columns and among
+ * different constraint types.
+ *
+ * XXX it might be useful to also list the column names in a PK, etc.
+ */
+ constr = new_objtree_VA("CONSTRAINT %{name}I %{definition}s",
+ 4,
+ "type", ObjTypeString, "constraint",
+ "contype", ObjTypeString, contype,
+ "name", ObjTypeString, NameStr(constrForm->conname),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrForm->oid));
+ elements = lappend(elements, new_object_object(constr));
+ }
+
+ systable_endscan(scan);
+ table_close(conRel, AccessShareLock);
+
+ return elements;
+}
+
+/*
+ * Deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
+{
+ ObjTree *oncommit;
+
+ oncommit = new_objtree("ON COMMIT");
+ switch (option)
+ {
+ case ONCOMMIT_DROP:
+ append_string_object(oncommit, "%{on_commit_value}s", "DROP");
+ break;
+
+ case ONCOMMIT_DELETE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "DELETE ROWS");
+ break;
+
+ case ONCOMMIT_PRESERVE_ROWS:
+ append_string_object(oncommit, "%{on_commit_value}s", "PRESERVE ROWS");
+ break;
+
+ case ONCOMMIT_NOOP:
+ append_null_object(oncommit, "%{on_commit_value}s");
+ append_bool_object(oncommit, "present", false);
+ break;
+ }
+
+ return oncommit;
+}
+
+/*
+ * Deparse DefElems, as used e.g. by ALTER COLUMN ... SET, into a list of SET
+ * (...) or RESET (...) contents.
+ */
+static ObjTree *
+deparse_DefElem(DefElem *elem, bool is_reset)
+{
+ ObjTree *set;
+ ObjTree *optname;
+
+ set = new_objtree("");
+ optname = new_objtree("");
+
+ if (elem->defnamespace != NULL)
+ append_string_object(optname, "%{schema}I.", elem->defnamespace);
+
+ append_string_object(optname, "%{label}I", elem->defname);
+
+ append_object_object(set, "%{label}s", optname);
+
+ if (!is_reset)
+ append_string_object(set, " = %{value}L",
+ elem->arg ? defGetString(elem) :
+ defGetBoolean(elem) ? "TRUE" : "FALSE");
+
+ return set;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_ColumnSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *colset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER COLUMN %{column}I RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "ALTER COLUMN %{column}I RESET ";
+ else
+ fmt = "ALTER COLUMN %{column}I SET ";
+
+ colset = new_objtree_VA(fmt, 1, "column", ObjTypeString, subcmd->name);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(colset, "(%{options:, }s)", sets);
+
+ return colset;
+}
+
+/*
+ * ... ALTER COLUMN ... SET/RESET (...)
+ */
+static ObjTree *
+deparse_RelSetOptions(AlterTableCmd *subcmd)
+{
+ List *sets = NIL;
+ ListCell *cell;
+ ObjTree *relset;
+ char *fmt;
+ bool is_reset = subcmd->subtype == AT_ResetRelOptions;
+
+ /*
+ * Verbose syntax
+ *
+ * RESET|SET (%{options:, }s)
+ */
+ if (is_reset)
+ fmt = "RESET ";
+ else
+ fmt = "SET ";
+
+ relset = new_objtree(fmt);
+
+ foreach(cell, (List *) subcmd->def)
+ {
+ DefElem *elem;
+ ObjTree *set;
+
+ elem = (DefElem *) lfirst(cell);
+ set = deparse_DefElem(elem, is_reset);
+ sets = lappend(sets, new_object_object(set));
+ }
+
+ Assert(sets);
+ append_array_object(relset, "(%{options:, }s)", sets);
+
+ return relset;
+}
+
+/*
+ * Deparse the inherits relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
+{
+ List *parents = NIL;
+ Relation inhRel;
+ SysScanDesc scan;
+ ScanKeyData key;
+ HeapTuple tuple;
+
+ inhRel = table_open(InheritsRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key,
+ Anum_pg_inherits_inhrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(objectId));
+
+ scan = systable_beginscan(inhRel, InheritsRelidSeqnoIndexId,
+ true, NULL, 1, &key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ ObjTree *parent;
+ Form_pg_inherits formInh = (Form_pg_inherits) GETSTRUCT(tuple);
+
+ parent = new_objtree_for_qualname_id(RelationRelationId,
+ formInh->inhparent);
+ parents = lappend(parents, new_object_object(parent));
+ }
+
+ systable_endscan(scan);
+ table_close(inhRel, RowExclusiveLock);
+
+ return parents;
+}
+
+/*
+ * Obtain the deparsed partition bound expression for the given table.
+ */
+static char *
+RelationGetPartitionBound(Oid relid)
+{
+ Datum deparsed;
+ Datum boundDatum;
+ bool isnull;
+ HeapTuple tuple;
+
+ tuple = SearchSysCache1(RELOID, relid);
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for relation %u",
+ relid);
+
+ boundDatum = SysCacheGetAttr(RELOID, tuple,
+ Anum_pg_class_relpartbound,
+ &isnull);
+
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(TextDatumGetCString(boundDatum)),
+ relid);
+
+ ReleaseSysCache(tuple);
+
+ return TextDatumGetCString(deparsed);
+}
+
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
+ char **index_am,
+ char **definition,
+ char **reloptions,
+ char **tablespace,
+ char **whereClause)
+{
+ HeapTuple ht_idx;
+ HeapTuple ht_idxrel;
+ HeapTuple ht_am;
+ Form_pg_index idxrec;
+ Form_pg_class idxrelrec;
+ Form_pg_am amrec;
+ IndexAmRoutine *amroutine;
+ List *indexprs;
+ ListCell *indexpr_item;
+ List *context;
+ Oid indrelid;
+ int keyno;
+ Datum indcollDatum;
+ Datum indclassDatum;
+ Datum indoptionDatum;
+ bool isnull;
+ oidvector *indcollation;
+ oidvector *indclass;
+ int2vector *indoption;
+ StringInfoData definitionBuf;
+
+ /*
+ * Fetch the pg_index tuple by the Oid of the index
+ */
+ ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idx))
+ elog(ERROR, "cache lookup failed for index %u", indexrelid);
+ idxrec = (Form_pg_index) GETSTRUCT(ht_idx);
+
+ indrelid = idxrec->indrelid;
+ Assert(indexrelid == idxrec->indexrelid);
+
+ /* Must get indcollation, indclass, and indoption the hard way */
+ indcollDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indcollation, &isnull);
+ Assert(!isnull);
+ indcollation = (oidvector *) DatumGetPointer(indcollDatum);
+
+ indclassDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indclass, &isnull);
+ Assert(!isnull);
+ indclass = (oidvector *) DatumGetPointer(indclassDatum);
+
+ indoptionDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indoption, &isnull);
+ Assert(!isnull);
+ indoption = (int2vector *) DatumGetPointer(indoptionDatum);
+
+ /*
+ * Fetch the pg_class tuple of the index relation
+ */
+ ht_idxrel = SearchSysCache1(RELOID, ObjectIdGetDatum(indexrelid));
+ if (!HeapTupleIsValid(ht_idxrel))
+ elog(ERROR, "cache lookup failed for relation %u", indexrelid);
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
+ ht_am = SearchSysCache1(AMOID, ObjectIdGetDatum(idxrelrec->relam));
+ if (!HeapTupleIsValid(ht_am))
+ elog(ERROR, "cache lookup failed for access method %u",
+ idxrelrec->relam);
+ amrec = (Form_pg_am) GETSTRUCT(ht_am);
+
+ /*
+ * Get the index expressions, if any. (NOTE: we do not use the relcache
+ * versions of the expressions and predicate, because we want to display
+ * non-const-folded expressions.)
+ */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indexprs, NULL))
+ {
+ Datum exprsDatum;
+ bool isnull;
+ char *exprsString;
+
+ exprsDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indexprs, &isnull);
+ Assert(!isnull);
+ exprsString = TextDatumGetCString(exprsDatum);
+ indexprs = (List *) stringToNode(exprsString);
+ pfree(exprsString);
+ }
+ else
+ indexprs = NIL;
+
+ indexpr_item = list_head(indexprs);
+
+ context = deparse_context_for(get_rel_name(indrelid), indrelid);
+
+ initStringInfo(&definitionBuf);
+
+ /* Output index AM */
+ *index_am = pstrdup(quote_identifier(NameStr(amrec->amname)));
+
+ /* Fetch the index AM's API struct */
+ amroutine = GetIndexAmRoutine(amrec->amhandler);
+
+ /*
+ * Output index definition. Note the outer parens must be supplied by
+ * caller.
+ */
+ for (keyno = 0; keyno < idxrec->indnatts; keyno++)
+ {
+ AttrNumber attnum = idxrec->indkey.values[keyno];
+ int16 opt = indoption->values[keyno];
+ Oid keycoltype;
+ Oid keycolcollation;
+ Oid indcoll;
+
+ appendStringInfoString(&definitionBuf, keyno == 0 ? "" : ", ");
+
+ if (attnum != 0)
+ {
+ /* Simple index column */
+ char *attname;
+ int32 keycoltypmod;
+
+ attname = get_attname(indrelid, attnum, false);
+ appendStringInfoString(&definitionBuf, quote_identifier(attname));
+ get_atttypetypmodcoll(indrelid, attnum,
+ &keycoltype, &keycoltypmod,
+ &keycolcollation);
+ }
+ else
+ {
+ /* Expressional index */
+ Node *indexkey;
+ char *str;
+
+ if (indexpr_item == NULL)
+ elog(ERROR, "too few entries in indexprs list");
+ indexkey = (Node *) lfirst(indexpr_item);
+ indexpr_item = lnext(indexprs, indexpr_item);
+ /* Deparse */
+ str = deparse_expression(indexkey, context, false, false);
+
+ /* Need parens if it's not a bare function call */
+ if (indexkey && IsA(indexkey, FuncExpr) &&
+ ((FuncExpr *) indexkey)->funcformat == COERCE_EXPLICIT_CALL)
+ appendStringInfoString(&definitionBuf, str);
+ else
+ appendStringInfo(&definitionBuf, "(%s)", str);
+
+ keycoltype = exprType(indexkey);
+ keycolcollation = exprCollation(indexkey);
+ }
+
+ /* Add collation, even if default */
+ indcoll = indcollation->values[keyno];
+ if (OidIsValid(indcoll))
+ appendStringInfo(&definitionBuf, " COLLATE %s",
+ generate_collation_name((indcoll)));
+
+ /* Add the operator class name, even if default */
+ get_opclass_name(indclass->values[keyno], InvalidOid, &definitionBuf);
+
+ /* Add options if relevant */
+ if (amroutine->amcanorder)
+ {
+ /* If it supports sort ordering, report DESC and NULLS opts */
+ if (opt & INDOPTION_DESC)
+ {
+ appendStringInfoString(&definitionBuf, " DESC");
+ /* NULLS FIRST is the default in this case */
+ if (!(opt & INDOPTION_NULLS_FIRST))
+ appendStringInfoString(&definitionBuf, " NULLS LAST");
+ }
+ else
+ {
+ if (opt & INDOPTION_NULLS_FIRST)
+ appendStringInfoString(&definitionBuf, " NULLS FIRST");
+ }
+ }
+
+ /* XXX excludeOps thingy was here; do we need anything? */
+ }
+ *definition = definitionBuf.data;
+
+ /* Output reloptions */
+ *reloptions = flatten_reloptions(indexrelid);
+
+ /* Output tablespace */
+ {
+ Oid tblspc;
+
+ tblspc = get_rel_tablespace(indexrelid);
+ if (OidIsValid(tblspc))
+ *tablespace = pstrdup(quote_identifier(get_tablespace_name(tblspc)));
+ else
+ *tablespace = NULL;
+ }
+
+ /* Report index predicate, if any */
+ if (!heap_attisnull(ht_idx, Anum_pg_index_indpred, NULL))
+ {
+ Node *node;
+ Datum predDatum;
+ bool isnull;
+ char *predString;
+
+ /* Convert text string to node tree */
+ predDatum = SysCacheGetAttr(INDEXRELID, ht_idx,
+ Anum_pg_index_indpred, &isnull);
+ Assert(!isnull);
+ predString = TextDatumGetCString(predDatum);
+ node = (Node *) stringToNode(predString);
+ pfree(predString);
+
+ /* Deparse */
+ *whereClause =
+ deparse_expression(node, context, false, false);
+ }
+ else
+ *whereClause = NULL;
+
+ /* Clean up */
+ ReleaseSysCache(ht_idx);
+ ReleaseSysCache(ht_idxrel);
+ ReleaseSysCache(ht_am);
+}
+
+/*
+ * Deparse an IndexStmt.
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
+static ObjTree *
+deparse_IndexStmt(Oid objectId, Node *parsetree)
+{
+ IndexStmt *node = (IndexStmt *) parsetree;
+ ObjTree *indexStmt;
+ ObjTree *tmp;
+ Relation idxrel;
+ Relation heaprel;
+ char *index_am;
+ char *definition;
+ char *reloptions;
+ char *tablespace;
+ char *whereClause;
+
+ if (node->primary || node->isconstraint)
+ {
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
+ return NULL;
+ }
+
+ idxrel = relation_open(objectId, AccessShareLock);
+ heaprel = relation_open(idxrel->rd_index->indrelid, AccessShareLock);
+
+ pg_get_indexdef_detailed(objectId,
+ &index_am, &definition, &reloptions,
+ &tablespace, &whereClause);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{unique}s INDEX %{concurrently}s %{if_not_exists}s %{name}I ON
+ * %{table}D USING %{index_am}s (%{definition}s) %{with}s %{tablespace}s
+ * %{where_clause}s
+ */
+ indexStmt = new_objtree("CREATE");
+
+ append_string_object(indexStmt, "%{unique}s",
+ node->unique ? "UNIQUE" : "");
+
+ append_format_string(indexStmt, "INDEX");
+
+ append_string_object(indexStmt, "%{concurrently}s",
+ node->concurrent ? "CONCURRENTLY" : "");
+
+ append_string_object(indexStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(indexStmt, "%{name}I",
+ RelationGetRelationName(idxrel));
+
+ append_object_object(indexStmt, "ON %{table}D",
+ new_objtree_for_qualname(heaprel->rd_rel->relnamespace,
+ RelationGetRelationName(heaprel)));
+
+ append_string_object(indexStmt, "USING %{index_am}s", index_am);
+
+ append_string_object(indexStmt, "(%{definition}s)", definition);
+
+ /* reloptions */
+ tmp = new_objtree("WITH");
+ if (reloptions)
+ append_string_object(tmp, "(%{opts}s)", reloptions);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{with}s", tmp);
+
+ /* tablespace */
+ tmp = new_objtree("TABLESPACE");
+ if (tablespace)
+ append_string_object(tmp, "%{tablespace}s", tablespace);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{tablespace}s", tmp);
+
+ /* WHERE clause */
+ tmp = new_objtree("WHERE");
+ if (whereClause)
+ append_string_object(tmp, "%{where}s", whereClause);
+ else
+ append_bool_object(tmp, "present", false);
+ append_object_object(indexStmt, "%{where_clause}s", tmp);
+
+ table_close(idxrel, AccessShareLock);
+ table_close(heaprel, AccessShareLock);
+
+ return indexStmt;
+}
+
+/*
+ * Deparse a CreateStmt (CREATE TABLE).
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateStmt(Oid objectId, Node *parsetree)
+{
+ CreateStmt *node = (CreateStmt *) parsetree;
+ Relation relation = relation_open(objectId, AccessShareLock);
+ List *dpcontext;
+ ObjTree *createStmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s TABLE %{if_not_exists}s %{identity}D [OF
+ * %{of_type}T | PARTITION OF %{parent_identity}D] %{table_elements}s
+ * %{inherits}s %{partition_by}s %{access_method}s %{with_clause}s
+ * %{on_commit}s %{tablespace}s
+ */
+ createStmt = new_objtree("CREATE");
+
+ append_string_object(createStmt, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ append_format_string(createStmt, "TABLE");
+
+ append_string_object(createStmt, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createStmt, "%{identity}D", tmp);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(relation),
+ objectId);
+
+ /*
+ * Typed tables and partitions use a slightly different format string: we
+ * must not put table_elements with parents directly in the fmt string,
+ * because if there are no options the parens must not be emitted; and
+ * also, typed tables do not allow for inheritance.
+ */
+ if (node->ofTypename || node->partbound)
+ {
+ List *tableelts = NIL;
+
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
+ if (node->ofTypename)
+ {
+ tmp = new_objtree_for_type(relation->rd_rel->reloftype, -1);
+ append_object_object(createStmt, "OF %{of_type}T", tmp);
+ }
+ else
+ {
+ List *parents;
+ ObjElem *elem;
+
+ parents = deparse_InhRelations(objectId);
+ elem = (ObjElem *) linitial(parents);
+
+ Assert(list_length(parents) == 1);
+
+ append_format_string(createStmt, "PARTITION OF");
+
+ append_object_object(createStmt, "%{parent_identity}D",
+ elem->value.object);
+ }
+
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ true, /* typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+ if (tableelts == NIL)
+ {
+ tmp = new_objtree("");
+ append_bool_object(tmp, "present", false);
+ }
+ else
+ tmp = new_objtree_VA("(%{elements:, }s)", 1,
+ "elements", ObjTypeArray, tableelts);
+ append_object_object(createStmt, "%{table_elements}s", tmp);
+ }
+ else
+ {
+ List *tableelts = NIL;
+
+ /*
+ * There is no need to process LIKE clauses separately; they have
+ * already been transformed into columns and constraints.
+ */
+
+ /*
+ * Process table elements: column definitions and constraints. Only
+ * the column definitions are obtained from the parse node itself. To
+ * get constraints we rely on pg_constraint, because the parse node
+ * might be missing some things such as the name of the constraints.
+ */
+ tableelts = deparse_TableElements(relation, node->tableElts, dpcontext,
+ false, /* not typed table */
+ false); /* not composite */
+ tableelts = obtainConstraints(tableelts, objectId, InvalidOid);
+
+ append_array_object(createStmt, "(%{table_elements:, }s)", tableelts);
+
+ /*
+ * Add inheritance specification. We cannot simply scan the list of
+ * parents from the parser node, because that may lack the actual
+ * qualified names of the parent relations. Rather than trying to
+ * re-resolve them from the information in the parse node, it seems
+ * more accurate and convenient to grab it from pg_inherits.
+ */
+ tmp = new_objtree("INHERITS");
+ if (list_length(node->inhRelations) > 0)
+ append_array_object(tmp, "(%{parents:, }D)", deparse_InhRelations(objectId));
+ else
+ {
+ append_null_object(tmp, "(%{parents:, }D)");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{inherits}s", tmp);
+ }
+
+ tmp = new_objtree("TABLESPACE");
+ if (node->tablespacename)
+ append_string_object(tmp, "%{tablespace}I", node->tablespacename);
+ else
+ {
+ append_null_object(tmp, "%{tablespace}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{tablespace}s", tmp);
+
+ append_object_object(createStmt, "%{on_commit}s",
+ deparse_OnCommitClause(node->oncommit));
+
+ /* FOR VALUES clause */
+ if (node->partbound)
+ {
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
+ append_string_object(createStmt, "%{partition_bound}s",
+ RelationGetPartitionBound(objectId));
+ }
+
+ /* PARTITION BY clause */
+ tmp = new_objtree("PARTITION BY");
+ if (relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "%{definition}s", pg_get_partkeydef_simple(objectId));
+ else
+ {
+ append_null_object(tmp, "%{definition}s");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{partition_by}s", tmp);
+
+ /* USING clause */
+ tmp = new_objtree("USING");
+ if (node->accessMethod)
+ append_string_object(tmp, "%{access_method}I", node->accessMethod);
+ else
+ {
+ append_null_object(tmp, "%{access_method}I");
+ append_bool_object(tmp, "present", false);
+ }
+ append_object_object(createStmt, "%{access_method}s", tmp);
+
+ /* WITH clause */
+ tmp = new_objtree("WITH");
+ list = NIL;
+
+ foreach(cell, node->options)
+ {
+ ObjTree *tmp2;
+ DefElem *opt = (DefElem *) lfirst(cell);
+
+ tmp2 = deparse_DefElem(opt, false);
+ list = lappend(list, new_object_object(tmp2));
+ }
+
+ if (list)
+ append_array_object(tmp, "(%{with:, }s)", list);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createStmt, "%{with_clause}s", tmp);
+
+ relation_close(relation, AccessShareLock);
+
+ return createStmt;
+}
+
+static ObjTree *
+deparse_FunctionSet(VariableSetKind kind, char *name, char *value)
+{
+ ObjTree *r;
+
+ if (kind == VAR_RESET_ALL)
+ {
+ r = new_objtree("RESET ALL");
+ }
+ else if (value != NULL)
+ {
+ r = new_objtree_VA("SET %{set_name}I", 1,
+ "set_name", ObjTypeString, name);
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be
+ * quoted.
+ */
+ if (GetConfigOptionFlags(name, true) & GUC_LIST_QUOTE)
+ append_string_object(r, "TO %{set_value}s", value);
+ else
+ append_string_object(r, "TO %{set_value}L", value);
+ }
+ else
+ {
+ r = new_objtree("RESET");
+ append_string_object(r, "%{set_name}I", name);
+ }
+
+ return r;
+}
+
+/*
+ * deparse_CreateFunctionStmt
+ * Deparse a CreateFunctionStmt (CREATE FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
+static ObjTree *
+deparse_CreateFunction(Oid objectId, Node *parsetree)
+{
+ CreateFunctionStmt *node = (CreateFunctionStmt *) parsetree;
+ ObjTree *createFunc;
+ ObjTree *tmp;
+ Datum tmpdatum;
+ char *source;
+ char *probin;
+ List *params;
+ List *defaults;
+ List *sets = NIL;
+ List *types = NIL;
+ ListCell *cell;
+ ListCell *curdef;
+ ListCell *table_params = NULL;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ HeapTuple langTup;
+ Oid *typarray;
+ Oid *trftypes;
+ Form_pg_language langForm;
+ int i;
+ int typnum;
+ int ntypes;
+ bool isnull;
+ bool isfunction;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /* get the corresponding pg_language tuple */
+ langTup = SearchSysCache1(LANGOID, procForm->prolang);
+ if (!HeapTupleIsValid(langTup))
+ elog(ERROR, "cache lookup failure for language with OID %u",
+ procForm->prolang);
+ langForm = (Form_pg_language) GETSTRUCT(langTup);
+
+ /*
+ * Determine useful values for prosrc and probin. We cope with probin
+ * being either NULL or "-", but prosrc must have a valid value.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_prosrc, &isnull);
+ if (isnull)
+ elog(ERROR, "null prosrc in function with OID %u", objectId);
+ source = TextDatumGetCString(tmpdatum);
+
+ /* Determine a useful value for probin */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_probin, &isnull);
+ if (isnull)
+ probin = NULL;
+ else
+ {
+ probin = TextDatumGetCString(tmpdatum);
+ if (probin[0] == '\0' || strcmp(probin, "-") == 0)
+ probin = NULL;
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{or_replace}s FUNCTION %{signature}s RETURNS %{return_type}s
+ * LANGUAGE %{transform_type}s %{language}I %{window}s %{volatility}s
+ * %{parallel_safety}s %{leakproof}s %{strict}s %{security_definer}s
+ * %{cost}s %{rows}s %{support}s %{set_options: }s AS %{objfile}L,
+ * %{symbol}L
+ */
+ createFunc = new_objtree("CREATE");
+
+ append_string_object(createFunc, "%{or_replace}s",
+ node->replace ? "OR REPLACE" : "");
+
+ /*
+ * To construct the arguments array, extract the type OIDs from the
+ * function's pg_proc entry. If pronargs equals the parameter list length,
+ * there are no OUT parameters and thus we can extract the type OID from
+ * proargtypes; otherwise we need to decode proallargtypes, which is
+ * a bit more involved.
+ */
+ typarray = palloc(list_length(node->parameters) * sizeof(Oid));
+ if (list_length(node->parameters) > procForm->pronargs)
+ {
+ bool isnull;
+ Datum alltypes;
+ Datum *values;
+ bool *nulls;
+ int nelems;
+
+ alltypes = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proallargtypes, &isnull);
+ if (isnull)
+ elog(ERROR, "NULL proallargtypes, but more parameters than args");
+ deconstruct_array(DatumGetArrayTypeP(alltypes),
+ OIDOID, 4, 't', 'i',
+ &values, &nulls, &nelems);
+ if (nelems != list_length(node->parameters))
+ elog(ERROR, "mismatched proallargatypes");
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = values[i];
+ }
+ else
+ {
+ for (i = 0; i < list_length(node->parameters); i++)
+ typarray[i] = procForm->proargtypes.values[i];
+ }
+
+ /*
+ * If there are any default expressions, we read the deparsed expression as
+ * a list so that we can attach them to each argument.
+ */
+ tmpdatum = SysCacheGetAttr(PROCOID, procTup,
+ Anum_pg_proc_proargdefaults, &isnull);
+ if (!isnull)
+ {
+ defaults = FunctionGetDefaults(DatumGetTextP(tmpdatum));
+ curdef = list_head(defaults);
+ }
+ else
+ {
+ defaults = NIL;
+ curdef = NULL;
+ }
+
+ /*
+ * Now iterate over each parameter in the parsetree to create the
+ * parameters array.
+ */
+ params = NIL;
+ typnum = 0;
+ foreach(cell, node->parameters)
+ {
+ FunctionParameter *param = (FunctionParameter *) lfirst(cell);
+ ObjTree *paramobj;
+ ObjTree *defaultval;
+ ObjTree *name;
+
+ /*
+ * A PARAM_TABLE parameter indicates end of input arguments; the
+ * following parameters are part of the return type. We ignore them
+ * here, but keep track of the current position in the list so that
+ * we can easily produce the return type below.
+ */
+ if (param->mode == FUNC_PARAM_TABLE)
+ {
+ table_params = cell;
+ break;
+ }
+
+ /*
+ * Verbose syntax for paramater: %{mode}s %{name}s %{type}T %{default}s
+ *
+ * Note that %{name}s is a string here, not an identifier; the reason
+ * for this is that an absent parameter name must produce an empty
+ * string, not "", which is what would happen if we were to use
+ * %{name}I here. So we add another level of indirection to allow us
+ * to inject a "present" parameter.
+ */
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{mode}s",
+ param->mode == FUNC_PARAM_IN ? "IN" :
+ param->mode == FUNC_PARAM_OUT ? "OUT" :
+ param->mode == FUNC_PARAM_INOUT ? "INOUT" :
+ param->mode == FUNC_PARAM_VARIADIC ? "VARIADIC" :
+ "IN");
+
+ /* optional wholesale suppression of "name" occurs here */
+
+ name = new_objtree("");
+ append_string_object(name, "%{name}I",
+ param->name ? param->name : "NULL");
+
+ append_bool_object(name, "present",
+ param->name ? true : false);
+
+ append_object_object(paramobj, "%{name}s", name);
+
+ defaultval = new_objtree("DEFAULT");
+ if (PointerIsValid(param->defexpr))
+ {
+ char *expr;
+
+ if (curdef == NULL)
+ elog(ERROR, "proargdefaults list too short");
+ expr = lfirst(curdef);
+
+ append_string_object(defaultval, "%{value}s", expr);
+ curdef = lnext(defaults, curdef);
+ }
+ else
+ append_bool_object(defaultval, "present", false);
+
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+
+ append_object_object(paramobj, "%{default}s", defaultval);
+
+ params = lappend(params, new_object_object(paramobj));
+ }
+
+ tmp = new_objtree_VA("%{identity}D", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{arguments:, }s", params);
+ append_format_string(tmp, ")");
+
+ isfunction = (procForm->prokind != PROKIND_PROCEDURE);
+
+ if (isfunction)
+ append_object_object(createFunc, "FUNCTION %{signature}s", tmp);
+ else
+ append_object_object(createFunc, "PROCEDURE %{signature}s", tmp);
+
+ /*
+ * A return type can adopt one of two forms: either a [SETOF] some_type, or
+ * a TABLE(list-of-types). We can tell the second form because we saw a
+ * table param above while scanning the argument list.
+ */
+ if (table_params == NULL)
+ {
+ tmp = new_objtree_VA("", 1,
+ "return_form", ObjTypeString, "plain");
+ append_string_object(tmp, "%{setof}s",
+ procForm->proretset ? "SETOF" : "");
+ append_object_object(tmp, "%{rettype}T",
+ new_objtree_for_type(procForm->prorettype, -1));
+ }
+ else
+ {
+ List *rettypes = NIL;
+ ObjTree *paramobj;
+
+ tmp = new_objtree_VA("TABLE", 1,
+ "return_form", ObjTypeString, "table");
+ for (; table_params != NULL; table_params = lnext(node->parameters, table_params))
+ {
+ FunctionParameter *param = lfirst(table_params);
+
+ paramobj = new_objtree("");
+ append_string_object(paramobj, "%{name}I", param->name);
+ append_object_object(paramobj, "%{type}T",
+ new_objtree_for_type(typarray[typnum++], -1));
+ rettypes = lappend(rettypes, new_object_object(paramobj));
+ }
+
+ append_array_object(tmp, "(%{rettypes:, }s)", rettypes);
+ }
+
+ if (isfunction)
+ append_object_object(createFunc, "RETURNS %{return_type}s", tmp);
+
+ /* TRANSFORM FOR TYPE */
+ tmp = new_objtree("TRANSFORM");
+
+ ntypes = get_func_trftypes(procTup, &trftypes);
+ for (i = 0; i < ntypes; i++)
+ {
+ tmp = new_objtree_VA("FOR TYPE %{type}T", 1,
+ "type", ObjTypeObject,
+ new_objtree_for_type(trftypes[i], -1));
+ types = lappend(types, tmp);
+ }
+
+ if (types)
+ append_array_object(tmp, "%{for_type:, }s", types);
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{transform_type}s", tmp);
+
+ append_string_object(createFunc, "LANGUAGE %{language}I",
+ NameStr(langForm->lanname));
+
+ if (isfunction)
+ {
+ append_string_object(createFunc, "%{window}s",
+ procForm->prokind == PROKIND_WINDOW ? "WINDOW" : "");
+ append_string_object(createFunc, "%{volatility}s",
+ procForm->provolatile == PROVOLATILE_VOLATILE ?
+ "VOLATILE" :
+ procForm->provolatile == PROVOLATILE_STABLE ?
+ "STABLE" :
+ procForm->provolatile == PROVOLATILE_IMMUTABLE ?
+ "IMMUTABLE" : "INVALID VOLATILITY");
+
+ append_string_object(createFunc, "%{parallel_safety}s",
+ procForm->proparallel == PROPARALLEL_SAFE ?
+ "PARALLEL SAFE" :
+ procForm->proparallel == PROPARALLEL_RESTRICTED ?
+ "PARALLEL RESTRICTED" : "PARALLEL UNSAFE");
+
+ append_string_object(createFunc, "%{leakproof}s",
+ procForm->proleakproof ? "LEAKPROOF" : "");
+ append_string_object(createFunc, "%{strict}s",
+ procForm->proisstrict ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT");
+
+ append_string_object(createFunc, "%{security_definer}s",
+ procForm->prosecdef ?
+ "SECURITY DEFINER" : "SECURITY INVOKER");
+
+ append_object_object(createFunc, "%{cost}s",
+ new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ procForm->procost));
+
+ tmp = new_objtree("ROWS");
+ if (procForm->prorows == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n", procForm->prorows);
+ append_object_object(createFunc, "%{rows}s", tmp);
+
+ tmp = new_objtree("SUPPORT %{name}s");
+ if (procForm->prosupport)
+ {
+ Oid argtypes[1];
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(createFunc, "%{support}s", tmp);
+ }
+
+ foreach(cell, node->options)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ sets = lappend(sets, new_object_object(tmp));
+ }
+ }
+ append_array_object(createFunc, "%{set_options: }s", sets);
+
+ /* Add the function definition */
+ (void) SysCacheGetAttr(PROCOID, procTup, Anum_pg_proc_prosqlbody, &isnull);
+ if (procForm->prolang == SQLlanguageId && !isnull)
+ {
+ StringInfoData buf;
+
+ initStringInfo(&buf);
+ print_function_sqlbody(&buf, procTup);
+
+ append_string_object(createFunc, "%{definition}s", buf.data);
+ }
+ else if (probin == NULL)
+ {
+ append_string_object(createFunc, "AS %{definition}L",
+ source);
+ }
+ else
+ {
+ append_string_object(createFunc, "AS %{objfile}L", probin);
+ append_string_object(createFunc, ", %{symbol}L", source);
+ }
+
+ ReleaseSysCache(langTup);
+ ReleaseSysCache(procTup);
+
+ return createFunc;
+}
+
+/*
+ * Deparse a AlterFunctionStmt (ALTER FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the alter command.
+ */
+static ObjTree *
+deparse_AlterFunction(Oid objectId, Node *parsetree)
+{
+ AlterFunctionStmt *node = (AlterFunctionStmt *) parsetree;
+ ObjTree *alterFunc;
+ ObjTree *sign;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ List *params;
+ List *elems = NIL;
+ ListCell *cell;
+ int i;
+
+ /* get the pg_proc tuple */
+ procTup = SearchSysCache1(PROCOID, objectId);
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failure for function with OID %u",
+ objectId);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER FUNCTION %{signature}s %{definition: }s
+ */
+ alterFunc = new_objtree_VA("ALTER FUNCTION", 0);
+
+ params = NIL;
+
+ /*
+ * ALTER FUNCTION does not change signature so we can use catalog
+ * to get input type Oids.
+ */
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *tmp = new_objtree_VA("%{type}T", 0);
+
+ append_object_object(tmp, "type",
+ new_objtree_for_type(procForm->proargtypes.values[i], -1));
+ params = lappend(params, new_object_object(tmp));
+ }
+
+ sign = new_objtree("");
+
+ append_object_object(sign, "%{identity}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ objectId));
+ append_array_object(sign, "(%{arguments:, }s)", params);
+
+ append_object_object(alterFunc, "%{signature}s", sign);
+
+ foreach(cell, node->actions)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(defel->defname, "volatility") == 0)
+ {
+ tmp = new_objtree_VA(strVal(defel->arg), 0);
+ }
+ else if (strcmp(defel->defname, "strict") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "RETURNS NULL ON NULL INPUT" :
+ "CALLED ON NULL INPUT", 0);
+ }
+ else if (strcmp(defel->defname, "security") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "SECURITY DEFINER" : "SECURITY INVOKER", 0);
+ }
+ else if (strcmp(defel->defname, "leakproof") == 0)
+ {
+ tmp = new_objtree_VA(intVal(defel->arg) ?
+ "LEAKPROOF" : "NOT LEAKPROOF", 0);
+ }
+ else if (strcmp(defel->defname, "cost") == 0)
+ {
+ tmp = new_objtree_VA("COST %{cost}n", 1,
+ "cost", ObjTypeFloat,
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "rows") == 0)
+ {
+ tmp = new_objtree_VA("ROWS", 0);
+ if (defGetNumeric(defel) == 0)
+ append_bool_object(tmp, "present", false);
+ else
+ append_float_object(tmp, "%{rows}n",
+ defGetNumeric(defel));
+ }
+ else if (strcmp(defel->defname, "set") == 0)
+ {
+ VariableSetStmt *sstmt = (VariableSetStmt *) defel->arg;
+ char *value = ExtractSetVariableArgs(sstmt);
+
+ tmp = deparse_FunctionSet(sstmt->kind, sstmt->name, value);
+ }
+ else if (strcmp(defel->defname, "support") == 0)
+ {
+ Oid argtypes[1];
+
+ tmp = new_objtree("SUPPORT");
+
+ Assert(procForm->prosupport);
+
+ /*
+ * We should qualify the support function's name if it wouldn't be
+ * resolved by lookup in the current search path.
+ */
+ argtypes[0] = INTERNALOID;
+ append_string_object(tmp, "%{name}s",
+ generate_function_name(procForm->prosupport, 1,
+ NIL, argtypes,
+ false, NULL,
+ EXPR_KIND_NONE));
+ }
+ else if (strcmp(defel->defname, "parallel") == 0)
+ {
+ char *fmt = psprintf("PARALLEL %s", strVal(defel->arg));
+ tmp = new_objtree(fmt);
+ }
+
+ elems = lappend(elems, new_object_object(tmp));
+ }
+
+ append_array_object(alterFunc, "%{definition: }s", elems);
+
+ ReleaseSysCache(procTup);
+
+ return alterFunc;
+}
+
+/*
+ * Deparse a AlterOwnerStmt (ALTER ... OWNER TO ...).
+ */
+static ObjTree *
+deparse_AlterOwnerStmt(ObjectAddress address, Node *parsetree)
+{
+ AlterOwnerStmt *node = (AlterOwnerStmt *) parsetree;
+ ObjTree *ownerStmt;
+ char *fmt;
+
+ fmt = psprintf("ALTER %s %%{identity}s OWNER TO %%{newowner}I",
+ stringify_objtype(node->objectType));
+
+ ownerStmt = new_objtree_VA(fmt, 2,
+ "identity", ObjTypeString,
+ getObjectIdentity(&address, false),
+ "newowner", ObjTypeString,
+ get_rolespec_name(node->newowner));
+
+ return ownerStmt;
+}
+
+/*
+ * Deparse a AlterOperatorStmt (ALTER OPERATOR ... SET ...).
+ */
+static ObjTree *
+deparse_AlterOperatorStmt(Oid objectId, Node *parsetree)
+{
+ HeapTuple oprTup;
+ AlterOperatorStmt *node = (AlterOperatorStmt *) parsetree;
+ ObjTree *alterop;
+ Form_pg_operator oprForm;
+ ListCell *cell;
+ List *list = NIL;
+
+ oprTup = SearchSysCache1(OPEROID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(oprTup))
+ elog(ERROR, "cache lookup failed for operator with OID %u", objectId);
+ oprForm = (Form_pg_operator) GETSTRUCT(oprTup);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER OPERATOR %{identity}O (%{left_type}T, %{right_type}T) SET
+ * (%{elems:, }s)
+ */
+ alterop = new_objtree_VA("ALTER OPERATOR %{identity}O", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(oprForm->oprnamespace,
+ NameStr(oprForm->oprname)));
+
+ /* LEFTARG */
+ if (OidIsValid(oprForm->oprleft))
+ append_object_object(alterop, "(%{left_type}T",
+ new_objtree_for_type(oprForm->oprleft, -1));
+ else
+ append_string_object(alterop, "(%{left_type}s", "NONE");
+
+ /* RIGHTARG */
+ Assert(OidIsValid(oprForm->oprleft));
+ append_object_object(alterop, ", %{right_type}T)",
+ new_objtree_for_type(oprForm->oprright, -1));
+
+ foreach(cell, node->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
+
+ if (strcmp(elem->defname, "restrict") == 0)
+ {
+ tmp = new_objtree_VA("RESTRICT=", 1,
+ "clause", ObjTypeString, "restrict");
+ if (OidIsValid(oprForm->oprrest))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprrest));
+ else
+ append_string_object(tmp, "%{procedure}s", "NONE");
+ }
+ else if (strcmp(elem->defname, "join") == 0)
+ {
+ tmp = new_objtree_VA("JOIN=", 1,
+ "clause", ObjTypeString, "join");
+ if (OidIsValid(oprForm->oprjoin))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprjoin));
+ else
+ append_string_object(tmp, "%{procedure}s", "NONE");
+ }
+
+ Assert(tmp);
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ append_array_object(alterop, "SET (%{elems:, }s)", list);
+
+ ReleaseSysCache(oprTup);
+
+ return alterop;
+}
+
+/*
+ * Return the given object type as a string.
+ */
+static const char *
+stringify_objtype(ObjectType objtype)
+{
+ switch (objtype)
+ {
+ case OBJECT_AGGREGATE:
+ return "AGGREGATE";
+ case OBJECT_CAST:
+ return "CAST";
+ case OBJECT_COLUMN:
+ return "COLUMN";
+ case OBJECT_COLLATION:
+ return "COLLATION";
+ case OBJECT_CONVERSION:
+ return "CONVERSION";
+ case OBJECT_DATABASE:
+ return "DATABASE";
+ case OBJECT_DOMAIN:
+ return "DOMAIN";
+ case OBJECT_EVENT_TRIGGER:
+ return "EVENT TRIGGER";
+ case OBJECT_EXTENSION:
+ return "EXTENSION";
+ case OBJECT_FDW:
+ return "FOREIGN DATA WRAPPER";
+ case OBJECT_FOREIGN_SERVER:
+ return "SERVER";
+ case OBJECT_FOREIGN_TABLE:
+ return "FOREIGN TABLE";
+ case OBJECT_FUNCTION:
+ return "FUNCTION";
+ case OBJECT_INDEX:
+ return "INDEX";
+ case OBJECT_LANGUAGE:
+ return "LANGUAGE";
+ case OBJECT_LARGEOBJECT:
+ return "LARGE OBJECT";
+ case OBJECT_MATVIEW:
+ return "MATERIALIZED VIEW";
+ case OBJECT_OPCLASS:
+ return "OPERATOR CLASS";
+ case OBJECT_OPERATOR:
+ return "OPERATOR";
+ case OBJECT_OPFAMILY:
+ return "OPERATOR FAMILY";
+ case OBJECT_ROLE:
+ return "ROLE";
+ case OBJECT_RULE:
+ return "RULE";
+ case OBJECT_SCHEMA:
+ return "SCHEMA";
+ case OBJECT_SEQUENCE:
+ return "SEQUENCE";
+ case OBJECT_TABLE:
+ return "TABLE";
+ case OBJECT_TABLESPACE:
+ return "TABLESPACE";
+ case OBJECT_TRIGGER:
+ return "TRIGGER";
+ case OBJECT_TSCONFIGURATION:
+ return "TEXT SEARCH CONFIGURATION";
+ /*
+ case OBJECT_TSCONFIG_MAPPING:
+ return "TEXT SEARCH CONFIGURATION MAPPING";
+ */
+ case OBJECT_TSDICTIONARY:
+ return "TEXT SEARCH DICTIONARY";
+ case OBJECT_TSPARSER:
+ return "TEXT SEARCH PARSER";
+ case OBJECT_TSTEMPLATE:
+ return "TEXT SEARCH TEMPLATE";
+ case OBJECT_TYPE:
+ return "TYPE";
+ case OBJECT_USER_MAPPING:
+ return "USER MAPPING";
+ case OBJECT_VIEW:
+ return "VIEW";
+
+ default:
+ elog(ERROR, "unsupported object type %d", objtype);
+ }
+}
+
+/*
+ * Deparse a RenameStmt.
+ */
+static ObjTree *
+deparse_RenameStmt(ObjectAddress address, Node *parsetree)
+{
+ RenameStmt *node = (RenameStmt *) parsetree;
+ ObjTree *renameStmt;
+ char *fmtstr;
+ const char *objtype;
+ Relation relation;
+ Oid schemaId;
+
+ /*
+ * In a ALTER .. RENAME command, we don't have the original name of the
+ * object in system catalogs: since we inspect them after the command has
+ * executed, the old name is already gone. Therefore, we extract it from
+ * the parse node. Note we still extract the schema name from the catalog
+ * (it might not be present in the parse node); it cannot possibly have
+ * changed anyway.
+ */
+ switch (node->renameType)
+ {
+ case OBJECT_TABLE:
+ case OBJECT_INDEX:
+ case OBJECT_SEQUENCE:
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER %s %{if_exists}s %{identity}D RENAME TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+ renameStmt = new_objtree_VA(fmtstr, 0);
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+ append_string_object(renameStmt, "RENAME TO %{newname}I",
+ node->newname);
+ relation_close(relation, AccessShareLock);
+ break;
+
+ case OBJECT_ATTRIBUTE:
+ case OBJECT_COLUMN:
+ relation = relation_open(address.objectId, AccessShareLock);
+ schemaId = RelationGetNamespace(relation);
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER objtype %{identity}D RENAME ATTRIBUTE
+ * %{colname}I TO %{newname}I %{cascade}s
+ *
+ * Normal types: ALTER objtype %{if_exists}s %%{identity}D RENAME
+ * COLUMN %{colname}I TO %{newname}I
+ */
+ objtype = stringify_objtype(node->renameType);
+ fmtstr = psprintf("ALTER %s", objtype);
+ renameStmt = new_objtree(fmtstr);
+
+ /* Composite types do not support IF EXISTS */
+ if (node->renameType == OBJECT_COLUMN)
+ append_string_object(renameStmt, "%{if_exists}s",
+ node->missing_ok ? "IF EXISTS" : "");
+
+ append_object_object(renameStmt, "%{identity}D",
+ new_objtree_for_qualname(schemaId,
+ node->relation->relname));
+
+ fmtstr = psprintf("RENAME %s %%{colname}I", objtype);
+
+ append_string_object(renameStmt, fmtstr, node->subname);
+ append_string_object(renameStmt, "TO %{newname}I", node->newname);
+
+ if (node->renameType == OBJECT_ATTRIBUTE)
+ append_object_object(renameStmt, "%{cascade}s",
+ new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool,
+ node->behavior == DROP_CASCADE));
+
+ relation_close(relation, AccessShareLock);
+
+ break;
+ case OBJECT_SCHEMA:
+ renameStmt =
+ new_objtree_VA("ALTER SCHEMA %{identity}I RENAME TO %{newname}I",
+ 1,
+ "identity", ObjTypeString, node->subname,
+ "newname", ObjTypeString, node->newname);
+ break;
+ case OBJECT_TRIGGER:
+ {
+ HeapTuple trigTup;
+ Form_pg_trigger trigForm;
+ Relation pg_trigger;
+
+ pg_trigger = relation_open(TriggerRelationId, AccessShareLock);
+ trigTup = get_catalog_object_by_oid(pg_trigger, get_object_attnum_oid(address.classId), address.objectId);
+ trigForm = (Form_pg_trigger) GETSTRUCT(trigTup);
+
+ renameStmt = new_objtree_VA("ALTER TRIGGER %{triggername}I ON %{identity}D RENAME TO %{newname}I",
+ 3,
+ "triggername", ObjTypeString, node->subname,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ trigForm->tgrelid),
+ "newname", ObjTypeString, node->newname);
+
+ relation_close(pg_trigger, AccessShareLock);
+ }
+ break;
+ default:
+ elog(ERROR, "unsupported object type %d", node->renameType);
+ }
+
+ return renameStmt;
+}
+
+/*
+ * Deparse the sequence CACHE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cache;
+ char *tmpstr;
+ char *fmt;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqcache);
+
+ if (alter_table)
+ fmt = "SET CACHE %{value}s";
+ else
+ fmt = "CACHE %{value}s";
+
+ cache = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cache",
+ "value", ObjTypeString, tmpstr);
+
+ return new_object_object(cache);
+}
+
+/*
+ * Deparse the sequence CYCLE option.
+ */
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *cycle;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET %{no}s CYCLE";
+ else
+ fmt = "%{no}s CYCLE";
+
+ cycle = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "cycle",
+ "no", ObjTypeString,
+ seqdata->seqcycle ? "" : "NO");
+ return new_object_object(cycle);
+}
+
+/*
+ * Deparse the sequence INCREMENT BY option.
+ */
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *incrementalby;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET INCREMENT BY %{value}s";
+ else
+ fmt = "INCREMENT BY %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqincrement);
+ incrementalby = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "seqincrement",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(incrementalby);
+}
+
+/*
+ * Deparse the sequence MINVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *minvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MINVALUE %{value}s";
+ else
+ fmt = "MINVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmin);
+ minvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "minvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(minvalue);
+}
+
+/*
+ * Deparse the sequence MAXVALUE option.
+ */
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *maxvalue;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET MAXVALUE %{value}s";
+ else
+ fmt = "MAXVALUE %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqmax);
+ maxvalue = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "maxvalue",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(maxvalue);
+}
+
+/*
+ * Deparse the sequence START WITH option.
+ */
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
+{
+ ObjTree *startwith;
+ char *tmpstr;
+ char *fmt;
+
+ if (alter_table)
+ fmt = "SET START WITH %{value}s";
+ else
+ fmt = "START WITH %{value}s";
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->seqstart);
+ startwith = new_objtree_VA(fmt, 2,
+ "clause", ObjTypeString, "start",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(startwith);
+}
+
+/*
+ * Deparse the sequence RESTART option.
+ */
+static inline ObjElem *
+deparse_Seq_Restart(ObjTree *parent, Form_pg_sequence_data seqdata)
+{
+ ObjTree *restart;
+ char *tmpstr;
+
+ tmpstr = psprintf(INT64_FORMAT, seqdata->last_value);
+ restart = new_objtree_VA("RESTART %{value}s", 2,
+ "clause", ObjTypeString, "restart",
+ "value", ObjTypeString, tmpstr);
+ return new_object_object(restart);
+}
+
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
+static ObjTree *
+deparse_CreateSchemaStmt(Oid objectId, Node *parsetree)
+{
+ CreateSchemaStmt *node = (CreateSchemaStmt *) parsetree;
+ ObjTree *createSchema;
+ ObjTree *auth;
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE SCHEMA %{if_not_exists}s %{name}I %{authorization}s
+ */
+ createSchema = new_objtree("CREATE SCHEMA");
+
+ append_string_object(createSchema, "%{if_not_exists}s",
+ node->if_not_exists ? "IF NOT EXISTS" : "");
+
+ append_string_object(createSchema, "%{name}I", node->schemaname);
+
+ auth = new_objtree("AUTHORIZATION");
+ if (node->authrole)
+ append_string_object(auth, "%{authorization_role}I",
+ get_rolespec_name(node->authrole));
+ else
+ {
+ append_null_object(auth, "%{authorization_role}I ");
+ append_bool_object(auth, "present", false);
+ }
+ append_object_object(createSchema, "%{authorization}s", auth);
+
+ return createSchema;
+}
+
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
+{
+ ObjTree *ownedby = NULL;
+ Relation depRel;
+ SysScanDesc scan;
+ ScanKeyData keys[3];
+ HeapTuple tuple;
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+ ScanKeyInit(&keys[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&keys[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(sequenceId));
+ ScanKeyInit(&keys[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(0));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, keys);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid ownerId;
+ Form_pg_depend depform;
+ ObjTree *tmp;
+ char *colname;
+
+ depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ /* only consider AUTO dependencies on pg_class */
+ if (depform->deptype != DEPENDENCY_AUTO)
+ continue;
+ if (depform->refclassid != RelationRelationId)
+ continue;
+ if (depform->refobjsubid <= 0)
+ continue;
+
+ ownerId = depform->refobjid;
+ colname = get_attname(ownerId, depform->refobjsubid, false);
+ if (colname == NULL)
+ continue;
+
+ tmp = new_objtree_for_qualname_id(RelationRelationId, ownerId);
+ append_string_object(tmp, "attrname", colname);
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 2,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeObject, tmp);
+ }
+
+ systable_endscan(scan);
+ relation_close(depRel, AccessShareLock);
+
+ /*
+ * If there's no owner column, emit an empty OWNED BY element, set up so
+ * that it won't print anything.
+ */
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
+}
+
+/*
+ * Deparse a CreateSeqStmt.
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *createSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ CreateSeqStmt *createSeqStmt = (CreateSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (createSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE %{persistence}s SEQUENCE %{identity}D
+ */
+ createSeq = new_objtree("CREATE");
+
+ append_string_object(createSeq, "%{persistence}s",
+ get_persistence_str(relation->rd_rel->relpersistence));
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(createSeq, "SEQUENCE %{identity}D", tmp);
+
+ /* definition elemets */
+ elems = lappend(elems, deparse_Seq_Cache(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Cycle(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_IncrementBy(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Minvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Maxvalue(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Startwith(createSeq, seqform, false));
+ elems = lappend(elems, deparse_Seq_Restart(createSeq, seqdata));
+ /* we purposefully do not emit OWNED BY here */
+
+ append_array_object(createSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return createSeq;
+}
+
+/*
+ * Deparse an AlterSeqStmt.
+ *
+ * Given a sequence OID and a parsetree that modified it, return an ObjTree
+ * representing the alter command.
+ */
+static ObjTree *
+deparse_AlterSeqStmt(Oid objectId, Node *parsetree)
+{
+ ObjTree *alterSeq;
+ ObjTree *tmp;
+ Relation relation;
+ Form_pg_sequence_data seqdata;
+ List *elems = NIL;
+ ListCell *cell;
+ Form_pg_sequence seqform;
+ Relation rel;
+ HeapTuple seqtuple;
+ AlterSeqStmt *alterSeqStmt = (AlterSeqStmt *) parsetree;
+
+ /*
+ * Sequence for IDENTITY COLUMN output separately(via CREATE TABLE or
+ * ALTER TABLE); return empty here.
+ */
+ if (alterSeqStmt->for_identity)
+ return NULL;
+
+ seqdata = get_sequence_values(objectId);
+
+ relation = relation_open(objectId, AccessShareLock);
+ rel = table_open(SequenceRelationId, RowExclusiveLock);
+ seqtuple = SearchSysCacheCopy1(SEQRELID,
+ ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(seqtuple))
+ elog(ERROR, "cache lookup failed for sequence %u",
+ objectId);
+
+ seqform = (Form_pg_sequence) GETSTRUCT(seqtuple);
+
+ alterSeq = new_objtree("ALTER SEQUENCE");
+
+ tmp = new_objtree_for_qualname(relation->rd_rel->relnamespace,
+ RelationGetRelationName(relation));
+ append_object_object(alterSeq, "%{identity}D", tmp);
+
+ foreach(cell, ((AlterSeqStmt *) parsetree)->options)
+ {
+ DefElem *elem = (DefElem *) lfirst(cell);
+ ObjElem *newelm;
+
+ if (strcmp(elem->defname, "cache") == 0)
+ newelm = deparse_Seq_Cache(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "cycle") == 0)
+ newelm = deparse_Seq_Cycle(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "increment") == 0)
+ newelm = deparse_Seq_IncrementBy(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "minvalue") == 0)
+ newelm = deparse_Seq_Minvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "maxvalue") == 0)
+ newelm = deparse_Seq_Maxvalue(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "start") == 0)
+ newelm = deparse_Seq_Startwith(alterSeq, seqform, false);
+ else if (strcmp(elem->defname, "restart") == 0)
+ newelm = deparse_Seq_Restart(alterSeq, seqdata);
+ else if (strcmp(elem->defname, "owned_by") == 0)
+ newelm = deparse_Seq_OwnedBy(alterSeq, objectId, false);
+ else
+ elog(ERROR, "invalid sequence option %s", elem->defname);
+
+ elems = lappend(elems, newelm);
+ }
+
+ append_array_object(alterSeq, "%{definition: }s", elems);
+
+ table_close(rel, RowExclusiveLock);
+ relation_close(relation, AccessShareLock);
+
+ return alterSeq;
+}
+
+/*
+ * Deparse all the collected subcommands and return an ObjTree representing the
+ * alter command.
+ */
+static ObjTree *
+deparse_AlterTableStmt(CollectedCommand *cmd)
+{
+ ObjTree *alterTableStmt;
+ ObjTree *tmp;
+ ObjTree *tmp2;
+ List *dpcontext;
+ Relation rel;
+ List *subcmds = NIL;
+ ListCell *cell;
+ char *fmtstr;
+ const char *reltype;
+ bool istype = false;
+ List *exprs = NIL;
+
+ Assert(cmd->type == SCT_AlterTable);
+
+ rel = relation_open(cmd->d.alterTable.objectId, AccessShareLock);
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ cmd->d.alterTable.objectId);
+
+ switch (rel->rd_rel->relkind)
+ {
+ case RELKIND_RELATION:
+ case RELKIND_PARTITIONED_TABLE:
+ reltype = "TABLE";
+ break;
+ case RELKIND_INDEX:
+ case RELKIND_PARTITIONED_INDEX:
+ reltype = "INDEX";
+ break;
+ case RELKIND_VIEW:
+ reltype = "VIEW";
+ break;
+ case RELKIND_COMPOSITE_TYPE:
+ reltype = "TYPE";
+ istype = true;
+ break;
+ case RELKIND_FOREIGN_TABLE:
+ reltype = "FOREIGN TABLE";
+ break;
+
+ /* TODO support for partitioned table */
+
+ default:
+ elog(ERROR, "unexpected relkind %d", rel->rd_rel->relkind);
+ }
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER reltype %{identity}D %{subcmds:, }s
+ */
+ fmtstr = psprintf("ALTER %s", reltype);
+ alterTableStmt = new_objtree(fmtstr);
+
+ tmp = new_objtree_for_qualname(rel->rd_rel->relnamespace,
+ RelationGetRelationName(rel));
+ append_object_object(alterTableStmt, "%{identity}D", tmp);
+
+ foreach(cell, cmd->d.alterTable.subcmds)
+ {
+ CollectedATSubcmd *sub = (CollectedATSubcmd *) lfirst(cell);
+ AlterTableCmd *subcmd = (AlterTableCmd *) sub->parsetree;
+ ObjTree *tree;
+
+ Assert(IsA(subcmd, AlterTableCmd));
+
+ switch (subcmd->subtype)
+ {
+ case AT_AddColumn:
+ case AT_AddColumnRecurse:
+ /* XXX need to set the "recurse" bit somewhere? */
+ Assert(IsA(subcmd->def, ColumnDef));
+ tree = deparse_ColumnDef(rel, dpcontext, false,
+ (ColumnDef *) subcmd->def, true, &exprs);
+ fmtstr = psprintf("ADD %s %%{definition}s",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "add column",
+ "definition", ObjTypeObject, tree);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndexConstraint:
+ {
+ IndexStmt *istmt;
+ Relation idx;
+ Oid constrOid = sub->address.objectId;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ Assert(istmt->isconstraint && istmt->unique);
+
+ idx = relation_open(istmt->indexOid, AccessShareLock);
+
+ /*
+ * Verbose syntax
+ *
+ * ADD CONSTRAINT %{name}I %{constraint_type}s USING INDEX
+ * %index_name}I %{deferrable}s %{init_deferred}s
+ */
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I",
+ 2,
+ "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid));
+
+ append_string_object(tmp, "%{constraint_type}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "USING INDEX %{index_name}I",
+ RelationGetRelationName(idx));
+ append_string_object(tmp, "%{deferrable}s", istmt->deferrable ?
+ "DEFERRABLE" : "NOT DEFERRABLE");
+ append_string_object(tmp, "%{init_deferred}s", istmt->initdeferred ?
+ "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_ReAddIndex:
+ case AT_ReAddConstraint:
+ case AT_ReAddComment:
+ case AT_ReplaceRelOptions:
+ /* Subtypes used for internal operations; nothing to do here */
+ break;
+
+ case AT_AddColumnToView:
+ /* CREATE OR REPLACE VIEW -- nothing to do here */
+ break;
+
+ case AT_ColumnDefault:
+ if (subcmd->def == NULL)
+ {
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP DEFAULT",
+ 2,
+ "type", ObjTypeString, "drop default",
+ "column", ObjTypeString, subcmd->name);
+ }
+ else
+ {
+ List *dpcontext;
+ HeapTuple attrtup;
+ AttrNumber attno;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET DEFAULT",
+ 2,
+ "type", ObjTypeString, "set default",
+ "column", ObjTypeString, subcmd->name);
+
+ dpcontext = deparse_context_for(RelationGetRelationName(rel),
+ RelationGetRelid(rel));
+ attrtup = SearchSysCacheAttName(RelationGetRelid(rel), subcmd->name);
+ attno = ((Form_pg_attribute) GETSTRUCT(attrtup))->attnum;
+ append_string_object(tmp, "%{definition}s",
+ RelationGetColumnDefault(rel, attno, dpcontext, NULL));
+ ReleaseSysCache(attrtup);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP NOT NULL",
+ 2,
+ "type", ObjTypeString, "drop not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetNotNull:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET NOT NULL",
+ 2,
+ "type", ObjTypeString, "set not null",
+ "column", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropExpression:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP EXPRESSION",
+ 2,
+ "type", ObjTypeString, "drop expression",
+ "column", ObjTypeString, subcmd->name);
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS": "");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetStatistics:
+ {
+ Assert(IsA(subcmd->def, Integer));
+ if (subcmd->name)
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeString, subcmd->name,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ else
+ tmp = new_objtree_VA("ALTER COLUMN %{column}n SET STATISTICS %{statistics}n",
+ 3,
+ "type", ObjTypeString, "set statistics",
+ "column", ObjTypeInteger, subcmd->num,
+ "statistics", ObjTypeInteger,
+ intVal((Integer *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_SetOptions:
+ case AT_ResetOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_ColumnSetOptions(subcmd)));
+ break;
+
+ case AT_SetStorage:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET STORAGE %{storage}s",
+ 3,
+ "type", ObjTypeString, "set storage",
+ "column", ObjTypeString, subcmd->name,
+ "storage", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetCompression:
+ Assert(IsA(subcmd->def, String));
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I SET COMPRESSION %{compression_method}s",
+ 3,
+ "type", ObjTypeString, "set compression",
+ "column", ObjTypeString, subcmd->name,
+ "compression_method", ObjTypeString,
+ strVal((String *) subcmd->def));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropColumnRecurse:
+ case AT_DropColumn:
+ fmtstr = psprintf("DROP %s %%{column}I",
+ istype ? "ATTRIBUTE" : "COLUMN");
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "drop column",
+ "column", ObjTypeString, subcmd->name);
+ tmp2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, subcmd->behavior);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddIndex:
+ {
+ Oid idxOid = sub->address.objectId;
+ IndexStmt *istmt;
+ Relation idx;
+ const char *idxname;
+ Oid constrOid;
+
+ Assert(IsA(subcmd->def, IndexStmt));
+ istmt = (IndexStmt *) subcmd->def;
+
+ if (!istmt->isconstraint)
+ break;
+
+ idx = relation_open(idxOid, AccessShareLock);
+ idxname = RelationGetRelationName(idx);
+
+ constrOid = get_relation_constraint_oid(
+ cmd->d.alterTable.objectId, idxname, false);
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, idxname,
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+
+ relation_close(idx, AccessShareLock);
+ }
+ break;
+
+ case AT_AddConstraint:
+ case AT_AddConstraintRecurse:
+ {
+ /* XXX need to set the "recurse" bit somewhere? */
+ Oid constrOid = sub->address.objectId;
+ bool isnull;
+ HeapTuple tup;
+ Datum val;
+ Constraint *constr;
+
+ Assert(IsA(subcmd->def, Constraint));
+ constr = castNode(Constraint, subcmd->def);
+
+ if (!constr->skip_validation)
+ {
+ tup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(constrOid));
+
+ if (HeapTupleIsValid(tup))
+ {
+ char *conbin;
+
+ /* Fetch constraint expression in parsetree form */
+ val = SysCacheGetAttr(CONSTROID, tup,
+ Anum_pg_constraint_conbin, &isnull);
+
+ if (!isnull)
+ {
+ conbin = TextDatumGetCString(val);
+ exprs = lappend(exprs, stringToNode(conbin));
+ }
+
+ ReleaseSysCache(tup);
+ }
+ }
+
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I %{definition}s",
+ 3,
+ "type", ObjTypeString, "add constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "definition", ObjTypeString,
+ pg_get_constraintdef_command_simple(constrOid));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_AlterConstraint:
+ {
+ Oid constrOid = sub->address.objectId;
+ Constraint *c = (Constraint *) subcmd->def;
+
+ /* If no constraint was altered, silently skip it */
+ if (!OidIsValid(constrOid))
+ break;
+
+ Assert(IsA(c, Constraint));
+ tmp = new_objtree_VA("ALTER CONSTRAINT %{name}I %{deferrable}s %{init_deferred}s",
+ 4,
+ "type", ObjTypeString, "alter constraint",
+ "name", ObjTypeString, get_constraint_name(constrOid),
+ "deferrable", ObjTypeString,
+ c->deferrable ? "DEFERRABLE" : "NOT DEFERRABLE",
+ "init_deferred", ObjTypeString,
+ c->initdeferred ? "INITIALLY DEFERRED" : "INITIALLY IMMEDIATE");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+ case AT_ValidateConstraintRecurse:
+ case AT_ValidateConstraint:
+ tmp = new_objtree_VA("VALIDATE CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "validate constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropConstraintRecurse:
+ case AT_DropConstraint:
+ tmp = new_objtree_VA("DROP CONSTRAINT %{constraint}I",
+ 2,
+ "type", ObjTypeString, "drop constraint",
+ "constraint", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AlterColumnType:
+ {
+ TupleDesc tupdesc = RelationGetDescr(rel);
+ Form_pg_attribute att;
+ ColumnDef *def;
+
+ att = &(tupdesc->attrs[sub->address.objectSubId - 1]);
+ def = (ColumnDef *) subcmd->def;
+ Assert(IsA(def, ColumnDef));
+
+ /*
+ * Verbose syntax
+ *
+ * Composite types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s ATTRIBUTE %{cascade}s
+ *
+ * Normal types: ALTER reltype %{column}I SET DATA TYPE
+ * %{datatype}T %{collation}s COLUMN %{using}s
+ */
+ fmtstr = psprintf("ALTER %s %%{column}I SET DATA TYPE",
+ istype ? "ATTRIBUTE" : "COLUMN");
+
+ tmp = new_objtree_VA(fmtstr, 2,
+ "type", ObjTypeString, "alter column type",
+ "column", ObjTypeString, subcmd->name);
+ /* Add the TYPE clause */
+ append_object_object(tmp, "%{datatype}T",
+ new_objtree_for_type(att->atttypid,
+ att->atttypmod));
+
+ /* Add a COLLATE clause, if needed */
+ tmp2 = new_objtree("COLLATE");
+ if (OidIsValid(att->attcollation))
+ {
+ ObjTree *collname;
+
+ collname = new_objtree_for_qualname_id(CollationRelationId,
+ att->attcollation);
+ append_object_object(tmp2, "%{name}D", collname);
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{collation}s", tmp2);
+
+ /* If not a composite type, add the USING clause */
+ if (!istype)
+ {
+ /*
+ * If there's a USING clause, transformAlterTableStmt
+ * ran it through transformExpr and stored the
+ * resulting node in cooked_default, which we can use
+ * here.
+ */
+ tmp2 = new_objtree("USING");
+ if (def->raw_default)
+ {
+ Datum deparsed;
+ char *defexpr;
+
+ exprs = lappend(exprs, def->cooked_default);
+ defexpr = nodeToString(def->cooked_default);
+ deparsed = DirectFunctionCall2(pg_get_expr,
+ CStringGetTextDatum(defexpr),
+ RelationGetRelid(rel));
+ append_string_object(tmp2, "%{expression}s",
+ TextDatumGetCString(deparsed));
+ }
+ else
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{using}s", tmp2);
+ }
+
+ /* If it's a composite type, add the CASCADE clause */
+ if (istype)
+ {
+ tmp2 = new_objtree("CASCADE");
+ if (subcmd->behavior != DROP_CASCADE)
+ append_bool_object(tmp2, "present", false);
+ append_object_object(tmp, "%{cascade}s", tmp2);
+ }
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+
+#ifdef TODOLIST
+ case AT_AlterColumnGenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def,
+ subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_ChangeOwner:
+ tmp = new_objtree_VA("OWNER TO %{owner}I", 2,
+ "type", ObjTypeString, "change owner",
+ "owner", ObjTypeString,
+ get_rolespec_name(subcmd->newowner));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ClusterOn:
+ tmp = new_objtree_VA("CLUSTER ON %{index}I", 2,
+ "type", ObjTypeString, "cluster on",
+ "index", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropCluster:
+ tmp = new_objtree_VA("SET WITHOUT CLUSTER", 1,
+ "type", ObjTypeString, "set without cluster");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetLogged:
+ tmp = new_objtree_VA("SET LOGGED", 1,
+ "type", ObjTypeString, "set logged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetUnLogged:
+ tmp = new_objtree_VA("SET UNLOGGED", 1,
+ "type", ObjTypeString, "set unlogged");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOids:
+ tmp = new_objtree_VA("SET WITHOUT OIDS", 1,
+ "type", ObjTypeString, "set without oids");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetAccessMethod:
+ tmp = new_objtree_VA("SET ACCESS METHOD %{access_method}I", 2,
+ "type", ObjTypeString, "set access method",
+ "access_method", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_SetTableSpace:
+ tmp = new_objtree_VA("SET TABLESPACE %{tablespace}I", 2,
+ "type", ObjTypeString, "set tablespace",
+ "tablespace", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_SetRelOptions:
+ case AT_ResetRelOptions:
+ subcmds = lappend(subcmds, new_object_object(
+ deparse_RelSetOptions(subcmd)));
+ break;
+
+ case AT_EnableTrig:
+ tmp = new_objtree_VA("ENABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysTrig:
+ tmp = new_objtree_VA("ENABLE ALWAYS TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable always trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaTrig:
+ tmp = new_objtree_VA("ENABLE REPLICA TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "enable replica trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrig:
+ tmp = new_objtree_VA("DISABLE TRIGGER %{trigger}I", 2,
+ "type", ObjTypeString, "disable trigger",
+ "trigger", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigAll:
+ tmp = new_objtree_VA("ENABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "enable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigAll:
+ tmp = new_objtree_VA("DISABLE TRIGGER ALL", 1,
+ "type", ObjTypeString, "disable trigger all");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableTrigUser:
+ tmp = new_objtree_VA("ENABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "enable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableTrigUser:
+ tmp = new_objtree_VA("DISABLE TRIGGER USER", 1,
+ "type", ObjTypeString, "disable trigger user");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRule:
+ tmp = new_objtree_VA("ENABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableAlwaysRule:
+ tmp = new_objtree_VA("ENABLE ALWAYS RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable always rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableReplicaRule:
+ tmp = new_objtree_VA("ENABLE REPLICA RULE %{rule}I", 2,
+ "type", ObjTypeString, "enable replica rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRule:
+ tmp = new_objtree_VA("DISABLE RULE %{rule}I", 2,
+ "type", ObjTypeString, "disable rule",
+ "rule", ObjTypeString, subcmd->name);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddInherit:
+ tmp = new_objtree_VA("INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropInherit:
+ tmp = new_objtree_VA("NO INHERIT %{parent}D", 2,
+ "type", ObjTypeString, "drop inherit",
+ "parent", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_AddOf:
+ tmp = new_objtree_VA("OF %{type_of}T", 2,
+ "type", ObjTypeString, "add of",
+ "type_of", ObjTypeObject,
+ new_objtree_for_type(sub->address.objectId, -1));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DropOf:
+ tmp = new_objtree_VA("NOT OF", 1,
+ "type", ObjTypeString, "not of");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_ReplicaIdentity:
+ tmp = new_objtree_VA("REPLICA IDENTITY", 1,
+ "type", ObjTypeString, "replica identity");
+ switch (((ReplicaIdentityStmt *) subcmd->def)->identity_type)
+ {
+ case REPLICA_IDENTITY_DEFAULT:
+ append_string_object(tmp, "%{ident}s", "DEFAULT");
+ break;
+ case REPLICA_IDENTITY_FULL:
+ append_string_object(tmp, "%{ident}s", "FULL");
+ break;
+ case REPLICA_IDENTITY_NOTHING:
+ append_string_object(tmp, "%{ident}s", "NOTHING");
+ break;
+ case REPLICA_IDENTITY_INDEX:
+ tmp2 = new_objtree_VA("USING INDEX %{index}I", 1,
+ "index", ObjTypeString,
+ ((ReplicaIdentityStmt *) subcmd->def)->name);
+ append_object_object(tmp, "%{ident}s", tmp2);
+ break;
+ }
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_EnableRowSecurity:
+ tmp = new_objtree_VA("ENABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "enable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+
+ case AT_DisableRowSecurity:
+ tmp = new_objtree_VA("DISABLE ROW LEVEL SECURITY", 1,
+ "type", ObjTypeString, "disable row security");
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#ifdef TODOLIST
+ case AT_GenericOptions:
+ tmp = deparse_FdwOptions((List *) subcmd->def, NULL);
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+#endif
+ case AT_AttachPartition:
+ tmp = new_objtree_VA("ATTACH PARTITION %{partition_identity}D",
+ 2,
+ "type", ObjTypeString, "attach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+
+ if (rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ append_string_object(tmp, "%{partition_bound}s",
+ RelationGetPartitionBound(sub->address.objectId));
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartition:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D",
+ 2,
+ "type", ObjTypeString, "detach partition",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_DetachPartitionFinalize:
+ tmp = new_objtree_VA("DETACH PARTITION %{partition_identity}D FINALIZE",
+ 2,
+ "type", ObjTypeString, "detach partition finalize",
+ "partition_identity", ObjTypeObject,
+ new_objtree_for_qualname_id(RelationRelationId,
+ sub->address.objectId));
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ case AT_AddIdentity:
+ {
+ AttrNumber attnum;
+ Oid seq_relid;
+ ObjTree *seqdef;
+ ColumnDef *coldef = (ColumnDef *) subcmd->def;
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I ADD %{identity_column}s",
+ 2,
+ "type", ObjTypeString, "add identity",
+ "column", ObjTypeString, subcmd->name);
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, coldef->identity, false);
+
+ append_object_object(tmp, "identity_column", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ }
+ break;
+ case AT_SetIdentity:
+ {
+ DefElem *defel;
+ char identity = 0;
+ ObjTree *seqdef;
+ AttrNumber attnum;
+ Oid seq_relid;
+
+
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I",
+ 2,
+ "type", ObjTypeString, "set identity",
+ "column", ObjTypeString, subcmd->name);
+
+ if (subcmd->def)
+ {
+ List *def = (List *) subcmd->def;
+
+ Assert(IsA(subcmd->def, List));
+
+ defel = linitial_node(DefElem, def);
+ identity = defGetInt32(defel);
+ }
+
+ attnum = get_attnum(RelationGetRelid(rel), subcmd->name);
+ seq_relid = getIdentitySequence(RelationGetRelid(rel), attnum, true);
+ seqdef = deparse_ColumnIdentity(seq_relid, identity, true);
+
+ append_object_object(tmp, "%{definition}s", seqdef);
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ }
+ case AT_DropIdentity:
+ tmp = new_objtree_VA("ALTER COLUMN %{column}I DROP IDENTITY",
+ 2,
+ "type", ObjTypeString, "drop identity",
+ "column", ObjTypeString, subcmd->name);
+
+ append_string_object(tmp, "%{if_not_exists}s",
+ subcmd->missing_ok ? "IF EXISTS" : "");
+
+ subcmds = lappend(subcmds, new_object_object(tmp));
+ break;
+ default:
+ elog(WARNING, "unsupported alter table subtype %d",
+ subcmd->subtype);
+ break;
+ }
+
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
+ if (contain_volatile_functions((Node *) exprs))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("ALTER TABLE command using volatile function cannot be replicated")));
+
+ /*
+ * Clean the list as we already confirmed there is no volatile
+ * function.
+ */
+ list_free(exprs);
+ exprs = NIL;
+ }
+
+ table_close(rel, AccessShareLock);
+
+ if (list_length(subcmds) == 0)
+ return NULL;
+
+ append_array_object(alterTableStmt, "%{subcmds:, }s", subcmds);
+ return alterTableStmt;
+}
+
+/*
+ * Deparse a CREATE OPERATOR CLASS command.
+ */
+static ObjTree *
+deparse_CreateOpClassStmt(CollectedCommand *cmd)
+{
+ Oid opcoid = cmd->d.createopc.address.objectId;
+ HeapTuple opcTup;
+ HeapTuple opfTup;
+ Form_pg_opfamily opfForm;
+ Form_pg_opclass opcForm;
+ ObjTree *stmt;
+ ObjTree *tmp;
+ List *list;
+ ListCell *cell;
+
+ opcTup = SearchSysCache1(CLAOID, ObjectIdGetDatum(opcoid));
+ if (!HeapTupleIsValid(opcTup))
+ elog(ERROR, "cache lookup failed for opclass with OID %u", opcoid);
+ opcForm = (Form_pg_opclass) GETSTRUCT(opcTup);
+
+ opfTup = SearchSysCache1(OPFAMILYOID, opcForm->opcfamily);
+ if (!HeapTupleIsValid(opfTup))
+ elog(ERROR, "cache lookup failed for operator family with OID %u", opcForm->opcfamily);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(opfTup);
+
+ /*
+ * Verbose syntax
+ *
+ * CREATE OPERATOR CLASS %{identity}D %{default}s FOR TYPE %{type}T USING
+ * %{amname}I %{opfamily}s AS %{items:, }s
+ */
+
+ stmt = new_objtree_VA("CREATE OPERATOR CLASS %{identity}D", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opcForm->opcnamespace,
+ NameStr(opcForm->opcname)));
+
+ /* Add the DEFAULT clause */
+ append_string_object(stmt, "%{default}s",
+ opcForm->opcdefault ? "DEFAULT" : "");
+
+ /* Add the FOR TYPE clause */
+ append_object_object(stmt, "FOR TYPE %{type}T",
+ new_objtree_for_type(opcForm->opcintype, -1));
+
+ /* Add the USING clause */
+ append_string_object(stmt, "USING %{amname}I", get_am_name(opcForm->opcmethod));
+
+ /*
+ * Add the FAMILY clause; but if it has the same name and namespace as the
+ * opclass, then have it expand to empty, because it would cause a failure
+ * if the opfamily was created internally.
+ */
+ tmp = new_objtree_VA("FAMILY %{opfamily}D", 1,
+ "opfamily", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)));
+
+ if (strcmp(NameStr(opfForm->opfname), NameStr(opcForm->opcname)) == 0 &&
+ opfForm->opfnamespace == opcForm->opcnamespace)
+ append_bool_object(tmp, "present", false);
+
+ append_object_object(stmt, "%{opfamily}s", tmp);
+
+ /*
+ * Add the initial item list. Note we always add the STORAGE clause, even
+ * when it is implicit in the original command.
+ */
+ tmp = new_objtree("STORAGE");
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(opcForm->opckeytype != InvalidOid ?
+ opcForm->opckeytype : opcForm->opcintype,
+ -1));
+ list = list_make1(new_object_object(tmp));
+
+ /* Add the declared operators */
+ foreach(cell, cmd->d.createopc.operators)
+ {
+ OpFamilyMember *oper = lfirst(cell);
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("OPERATOR %{num}n %{operator}O(%{ltype}T, %{rtype}T)",
+ 4,
+ "num", ObjTypeInteger, oper->number,
+ "operator", ObjTypeObject,
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oper->object),
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(oper->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(oper->righttype, -1));
+
+ /* Add the FOR SEARCH / FOR ORDER BY clause */
+ if (oper->sortfamily == InvalidOid)
+ append_string_object(tmp, "%{purpose}s", "FOR SEARCH");
+ else
+ {
+ ObjTree *tmp2;
+
+ tmp2 = new_objtree_VA("FOR ORDER BY %{opfamily}D", 0);
+ append_object_object(tmp2, "opfamily",
+ new_objtree_for_qualname_id(OperatorFamilyRelationId,
+ oper->sortfamily));
+ append_object_object(tmp, "%{purpose}s", tmp2);
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ /* Add the declared support functions */
+ foreach(cell, cmd->d.createopc.procedures)
+ {
+ OpFamilyMember *proc = lfirst(cell);
+ ObjTree *tmp;
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ Oid *proargtypes;
+ List *arglist;
+ int i;
+
+ procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(proc->object));
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failed for procedure %u", proc->object);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ tmp = new_objtree_VA("FUNCTION %{num}n (%{ltype}T, %{rtype}T) %{function}D",
+ 4,
+ "num", ObjTypeInteger, proc->number,
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(proc->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(proc->righttype, -1),
+ "function", ObjTypeObject,
+ new_objtree_for_qualname(procForm->pronamespace,
+ NameStr(procForm->proname)));
+
+ proargtypes = procForm->proargtypes.values;
+ arglist = NIL;
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *arg;
+
+ arg = new_objtree_for_type(proargtypes[i], -1);
+ arglist = lappend(arglist, new_object_object(arg));
+ }
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{argtypes:, }T", arglist);
+ append_format_string(tmp, ")");
+
+ ReleaseSysCache(procTup);
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ append_array_object(stmt, "AS %{items:, }s", list);
+
+ ReleaseSysCache(opfTup);
+ ReleaseSysCache(opcTup);
+
+ return stmt;
+}
+
+/*
+ * Deparse a CreateTrigStmt (CREATE OPERATOR FAMILY)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_CreateOpFamily(Oid objectId, Node *parsetree)
+{
+ HeapTuple opfTup;
+ HeapTuple amTup;
+ Form_pg_opfamily opfForm;
+ Form_pg_am amForm;
+ ObjTree *copfStmt;
+
+ opfTup = SearchSysCache1(OPFAMILYOID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(opfTup))
+ elog(ERROR, "cache lookup failed for operator family with OID %u", objectId);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(opfTup);
+
+ amTup = SearchSysCache1(AMOID, ObjectIdGetDatum(opfForm->opfmethod));
+ if (!HeapTupleIsValid(amTup))
+ elog(ERROR, "cache lookup failed for access method %u",
+ opfForm->opfmethod);
+ amForm = (Form_pg_am) GETSTRUCT(amTup);
+
+ copfStmt = new_objtree_VA("CREATE OPERATOR FAMILY %{identity}D USING %{amname}I",
+ 2,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)),
+ "amname", ObjTypeString, NameStr(amForm->amname));
+
+ ReleaseSysCache(amTup);
+ ReleaseSysCache(opfTup);
+
+ return copfStmt;
+}
+
+/*
+ * Deparse an ALTER OPERATOR FAMILY ADD/DROP command.
+ */
+static ObjTree *
+deparse_AlterOpFamily(CollectedCommand *cmd)
+{
+ ObjTree *alterOpFam;
+ AlterOpFamilyStmt *stmt = (AlterOpFamilyStmt *) cmd->parsetree;
+ HeapTuple ftp;
+ Form_pg_opfamily opfForm;
+ List *list;
+ ListCell *cell;
+
+ ftp = SearchSysCache1(OPFAMILYOID,
+ ObjectIdGetDatum(cmd->d.opfam.address.objectId));
+ if (!HeapTupleIsValid(ftp))
+ elog(ERROR, "cache lookup failed for operator family %u",
+ cmd->d.opfam.address.objectId);
+ opfForm = (Form_pg_opfamily) GETSTRUCT(ftp);
+
+ /*
+ * Verbose syntax
+ *
+ * ALTER OPERATOR FAMILY %{identity}D USING %{amname}I ADD/DROP
+ * %{items:, }s
+ */
+ alterOpFam = new_objtree_VA("ALTER OPERATOR FAMILY %{identity}D "
+ "USING %{amname}I", 2,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(opfForm->opfnamespace,
+ NameStr(opfForm->opfname)),
+ "amname", ObjTypeString, stmt->amname);
+
+ list = NIL;
+ foreach(cell, cmd->d.opfam.operators)
+ {
+ OpFamilyMember *oper = lfirst(cell);
+ ObjTree *tmp;
+
+ /*
+ * Verbose syntax
+ *
+ * OPERATOR %{num}n %{operator}O(%{ltype}T, %{rtype}T) %{purpose}s
+ */
+ tmp = new_objtree_VA("OPERATOR %{num}n", 1,
+ "num", ObjTypeInteger, oper->number);
+
+ /* Add the operator name; the DROP case doesn't have this */
+ if (!stmt->isDrop)
+ {
+ append_object_object(tmp, "%{operator}O",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oper->object));
+ }
+
+ /* Add the types */
+ append_object_object(tmp, "(%{ltype}T,",
+ new_objtree_for_type(oper->lefttype, -1));
+ append_object_object(tmp, "%{rtype}T)",
+ new_objtree_for_type(oper->righttype, -1));
+
+ /* Add the FOR SEARCH / FOR ORDER BY clause; not in the DROP case */
+ if (!stmt->isDrop)
+ {
+ if (oper->sortfamily == InvalidOid)
+ append_string_object(tmp, "%{purpose}s", "FOR SEARCH");
+ else
+ {
+ ObjTree *tmp2;
+
+ tmp2 = new_objtree_VA("FOR ORDER BY", 0);
+ append_object_object(tmp2, "%{opfamily}D",
+ new_objtree_for_qualname_id(OperatorFamilyRelationId,
+ oper->sortfamily));
+ append_object_object(tmp, "%{purpose}s", tmp2);
+ }
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ foreach(cell, cmd->d.opfam.procedures)
+ {
+ OpFamilyMember *proc = lfirst(cell);
+ ObjTree *tmp;
+
+ tmp = new_objtree_VA("FUNCTION %{num}n (%{ltype}T, %{rtype}T)", 3,
+ "num", ObjTypeInteger, proc->number,
+ "ltype", ObjTypeObject,
+ new_objtree_for_type(proc->lefttype, -1),
+ "rtype", ObjTypeObject,
+ new_objtree_for_type(proc->righttype, -1));
+
+ /* Add the function name and arg types; the DROP case doesn't have this */
+ if (!stmt->isDrop)
+ {
+ HeapTuple procTup;
+ Form_pg_proc procForm;
+ Oid *proargtypes;
+ List *arglist;
+ int i;
+
+ procTup = SearchSysCache1(PROCOID, ObjectIdGetDatum(proc->object));
+ if (!HeapTupleIsValid(procTup))
+ elog(ERROR, "cache lookup failed for procedure %u", proc->object);
+ procForm = (Form_pg_proc) GETSTRUCT(procTup);
+
+ proargtypes = procForm->proargtypes.values;
+ arglist = NIL;
+ for (i = 0; i < procForm->pronargs; i++)
+ {
+ ObjTree *arg;
+
+ arg = new_objtree_for_type(proargtypes[i], -1);
+ arglist = lappend(arglist, new_object_object(arg));
+ }
+
+ append_object_object(tmp, "%{function}D",
+ new_objtree_for_qualname(procForm->pronamespace,
+ NameStr(procForm->proname)));
+
+ append_format_string(tmp, "(");
+ append_array_object(tmp, "%{argtypes:, }T", arglist);
+ append_format_string(tmp, ")");
+
+ ReleaseSysCache(procTup);
+ }
+
+ list = lappend(list, new_object_object(tmp));
+ }
+
+ if (!stmt->isDrop)
+ append_format_string(alterOpFam, "ADD");
+ else
+ append_format_string(alterOpFam, "DROP");
+
+ append_array_object(alterOpFam, "%{items:, }s", list);
+
+ ReleaseSysCache(ftp);
+
+ return alterOpFam;
+}
+
+/*
+ * Deparse a DefineStmt (CREATE OPERATOR)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
+static ObjTree *
+deparse_DefineStmt_Operator(Oid objectId, DefineStmt *define)
+{
+ HeapTuple oprTup;
+ ObjTree *stmt;
+ ObjTree *tmp;
+ List *list;
+ Form_pg_operator oprForm;
+
+ oprTup = SearchSysCache1(OPEROID, ObjectIdGetDatum(objectId));
+ if (!HeapTupleIsValid(oprTup))
+ elog(ERROR, "cache lookup failed for operator with OID %u", objectId);
+ oprForm = (Form_pg_operator) GETSTRUCT(oprTup);
+
+ stmt = new_objtree_VA("CREATE OPERATOR %{identity}O", 1,
+ "identity", ObjTypeObject,
+ new_objtree_for_qualname(oprForm->oprnamespace,
+ NameStr(oprForm->oprname)));
+
+ /* Add the definition clause */
+ list = NIL;
+
+ /* PROCEDURE */
+ tmp = new_objtree_VA("PROCEDURE=%{procedure}D", 2,
+ "clause", ObjTypeString, "procedure",
+ "procedure", ObjTypeObject,
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprcode));
+ list = lappend(list, new_object_object(tmp));
+
+ /* LEFTARG */
+ tmp = new_objtree_VA("LEFTARG=", 1,
+ "clause", ObjTypeString, "leftarg");
+ if (OidIsValid(oprForm->oprleft))
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(oprForm->oprleft, -1));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* RIGHTARG */
+ tmp = new_objtree_VA("RIGHTARG=", 1,
+ "clause", ObjTypeString, "rightarg");
+ if (OidIsValid(oprForm->oprright))
+ append_object_object(tmp, "%{type}T",
+ new_objtree_for_type(oprForm->oprright, -1));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* COMMUTATOR */
+ tmp = new_objtree_VA("COMMUTATOR=", 1,
+ "clause", ObjTypeString, "commutator");
+ if (OidIsValid(oprForm->oprcom))
+ append_object_object(tmp, "%{oper}D",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oprForm->oprcom));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* NEGATOR */
+ tmp = new_objtree_VA("NEGATOR=", 1,
+ "clause", ObjTypeString, "negator");
+ if (OidIsValid(oprForm->oprnegate))
+ append_object_object(tmp, "%{oper}D",
+ new_objtree_for_qualname_id(OperatorRelationId,
+ oprForm->oprnegate));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* RESTRICT */
+ tmp = new_objtree_VA("RESTRICT=", 1,
+ "clause", ObjTypeString, "restrict");
+ if (OidIsValid(oprForm->oprrest))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprrest));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* JOIN */
+ tmp = new_objtree_VA("JOIN=", 1,
+ "clause", ObjTypeString, "join");
+ if (OidIsValid(oprForm->oprjoin))
+ append_object_object(tmp, "%{procedure}D",
+ new_objtree_for_qualname_id(ProcedureRelationId,
+ oprForm->oprjoin));
+ else
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* MERGES */
+ tmp = new_objtree_VA("MERGES", 1,
+ "clause", ObjTypeString, "merges");
+ if (!oprForm->oprcanmerge)
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ /* HASHES */
+ tmp = new_objtree_VA("HASHES", 1,
+ "clause", ObjTypeString, "hashes");
+ if (!oprForm->oprcanhash)
+ append_bool_object(tmp, "present", false);
+ list = lappend(list, new_object_object(tmp));
+
+ append_array_object(stmt, "(%{elems:, }s)", list);
+
+ ReleaseSysCache(oprTup);
+
+ return stmt;
+}
+
+/*
+ * Deparse a DefineStmt.
+ */
+static ObjTree *
+deparse_DefineStmt(Oid objectId, Node *parsetree, ObjectAddress secondaryObj)
+{
+ DefineStmt *define = (DefineStmt *) parsetree;
+ ObjTree *defStmt;
+
+ switch (define->kind)
+ {
+ case OBJECT_OPERATOR:
+ defStmt = deparse_DefineStmt_Operator(objectId, define);
+ break;
+
+ default:
+ elog(ERROR, "unsupported object kind");
+ return NULL;
+ }
+
+ return defStmt;
+}
+
+/*
+ * Handle deparsing of simple commands.
+ *
+ * This function should cover all cases handled in ProcessUtilitySlow.
+ */
+static ObjTree *
+deparse_simple_command(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+ ObjTree *command;
+
+ Assert(cmd->type == SCT_Simple);
+
+ parsetree = cmd->parsetree;
+ objectId = cmd->d.simple.address.objectId;
+
+ /* This switch needs to handle everything that ProcessUtilitySlow does */
+ switch (nodeTag(parsetree))
+ {
+ case T_CreateSchemaStmt:
+ command = deparse_CreateSchemaStmt(objectId, parsetree);
+ break;
+
+ case T_CreateStmt:
+ command = deparse_CreateStmt(objectId, parsetree);
+ break;
+
+ case T_CreateTrigStmt:
+ command = deparse_CreateTrigStmt(objectId, parsetree);
+ break;
+
+ case T_CreateSeqStmt:
+ command = deparse_CreateSeqStmt(objectId, parsetree);
+ break;
+
+ case T_AlterSeqStmt:
+ command = deparse_AlterSeqStmt(objectId, parsetree);
+ break;
+
+ case T_DefineStmt:
+ command = deparse_DefineStmt(objectId, parsetree,
+ cmd->d.simple.secondaryObject);
+ break;
+
+ case T_CreateOpFamilyStmt:
+ command = deparse_CreateOpFamily(objectId, parsetree);
+ break;
+
+ case T_IndexStmt:
+ command = deparse_IndexStmt(objectId, parsetree);
+ break;
+
+ case T_CreateFunctionStmt:
+ command = deparse_CreateFunction(objectId, parsetree);
+ break;
+
+ case T_AlterFunctionStmt:
+ command = deparse_AlterFunction(objectId, parsetree);
+ break;
+
+ case T_RenameStmt:
+ command = deparse_RenameStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_AlterOwnerStmt:
+ command = deparse_AlterOwnerStmt(cmd->d.simple.address, parsetree);
+ break;
+
+ case T_AlterOperatorStmt:
+ command = deparse_AlterOperatorStmt(objectId, parsetree);
+ break;
+ default:
+ command = NULL;
+ elog(LOG, "unrecognized node type in deparse command: %d",
+ (int) nodeTag(parsetree));
+ }
+
+ return command;
+}
+
+/*
+ * Handle deparsing of DROP commands.
+ */
+char *
+deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior)
+{
+ StringInfoData str;
+ char *command;
+ char *identity = (char *) objidentity;
+ char *fmt;
+ ObjTree *stmt, *stmt2;
+ Jsonb *jsonb;
+
+ initStringInfo(&str);
+
+ fmt = psprintf("DROP %s IF EXISTS %%{objidentity}s", objecttype);
+
+ stmt = new_objtree_VA(fmt, 1, "objidentity", ObjTypeString, identity);
+ stmt2 = new_objtree_VA("CASCADE", 1,
+ "present", ObjTypeBool, behavior == DROP_CASCADE);
+
+ append_object_object(stmt, "%{cascade}s", stmt2);
+
+ jsonb = objtree_to_jsonb(stmt);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+
+ return command;
+}
+
+/*
+ * Workhorse to deparse a CollectedCommand.
+ */
+char *
+deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
+{
+ OverrideSearchPath *overridePath;
+ MemoryContext oldcxt;
+ MemoryContext tmpcxt;
+ ObjTree *tree;
+ char *command;
+ StringInfoData str;
+
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
+ initStringInfo(&str);
+ tmpcxt = AllocSetContextCreate(CurrentMemoryContext,
+ "deparse ctx",
+ ALLOCSET_DEFAULT_MINSIZE,
+ ALLOCSET_DEFAULT_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+ oldcxt = MemoryContextSwitchTo(tmpcxt);
+
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
+ overridePath = GetOverrideSearchPath(CurrentMemoryContext);
+ overridePath->schemas = NIL;
+ overridePath->addCatalog = false;
+ overridePath->addTemp = true;
+ PushOverrideSearchPath(overridePath);
+
+ verbose = verbose_mode;
+
+ switch (cmd->type)
+ {
+ case SCT_Simple:
+ tree = deparse_simple_command(cmd);
+ break;
+ case SCT_AlterTable:
+ tree = deparse_AlterTableStmt(cmd);
+ break;
+ case SCT_AlterOpFamily:
+ tree = deparse_AlterOpFamily(cmd);
+ break;
+ case SCT_CreateOpClass:
+ tree = deparse_CreateOpClassStmt(cmd);
+ break;
+ default:
+ elog(ERROR, "unexpected deparse node type %d", cmd->type);
+ }
+
+ PopOverrideSearchPath();
+
+ if (tree)
+ {
+ Jsonb *jsonb;
+
+ jsonb = objtree_to_jsonb(tree);
+ command = JsonbToCString(&str, &jsonb->root, JSONB_ESTIMATED_LEN);
+ }
+ else
+ command = NULL;
+
+ /*
+ * Clean up. Note that since we created the StringInfo in the caller's
+ * context, the output string is not deleted here.
+ */
+ MemoryContextSwitchTo(oldcxt);
+ MemoryContextDelete(tmpcxt);
+
+ return command;
+}
+
+/*
+ * Given a CollectedCommand, return a JSON representation of it.
+ *
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
+{
+ CollectedCommand *cmd = (CollectedCommand *) PG_GETARG_POINTER(0);
+ char *command;
+
+ command = deparse_utility_command(cmd, true);
+
+ if (command)
+ PG_RETURN_TEXT_P(CStringGetTextDatum(command));
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/commands/ddl_json.c b/src/backend/commands/ddl_json.c
new file mode 100644
index 0000000..d64719b
--- /dev/null
+++ b/src/backend/commands/ddl_json.c
@@ -0,0 +1,760 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_json.c
+ * JSON code related to DDL command deparsing
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/commands/ddl_json.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "lib/stringinfo.h"
+#include "utils/builtins.h"
+#include "utils/jsonb.h"
+
+
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
+typedef enum
+{
+ SpecTypename,
+ SpecOperatorname,
+ SpecDottedName,
+ SpecString,
+ SpecNumber,
+ SpecStringLiteral,
+ SpecIdentifier,
+ SpecRole
+} convSpecifier;
+
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
+
+static bool expand_one_jsonb_element(StringInfo buf, char *param,
+ JsonbValue *jsonval, convSpecifier specifier,
+ const char *fmt);
+static void expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep,
+ convSpecifier specifier, const char *fmt);
+static void fmtstr_error_callback(void *arg);
+char *ddl_deparse_json_to_string(char *jsonb);
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvBool, an error is raised. If it doesn't
+ * exist, tv_absent is returned; otherwise return the actual trivalue.
+ */
+static trivalue
+find_bool_in_jsonbcontainer(JsonbContainer *container, char *keyname)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ trivalue result;
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ return tv_absent;
+ if (value->type != jbvBool)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not of type boolean",
+ keyname)));
+ result = value->val.boolean ? tv_true : tv_false;
+ pfree(value);
+
+ return result;
+}
+
+/*
+ * Given a JsonbContainer, find the JsonbValue with the given key name in it.
+ * If it's of a type other than jbvString, an error is raised. If it doesn't
+ * exist, an error is raised unless missing_ok; otherwise return NULL.
+ *
+ * If it exists and is a string, a freshly palloc'ed copy is returned.
+ *
+ * If *length is not NULL, it is set to the length of the string.
+ */
+static char *
+find_string_in_jsonbcontainer(JsonbContainer *container, char *keyname,
+ bool missing_ok, int *length)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ char *str;
+
+ /* XXX verify that this is an object, not an array */
+
+ key.type = jbvString;
+ key.val.string.val = keyname;
+ key.val.string.len = strlen(keyname);
+ value = findJsonbValueFromContainer(container,
+ JB_FOBJECT, &key);
+ if (value == NULL)
+ {
+ if (missing_ok)
+ return NULL;
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing element \"%s\" in json object", keyname)));
+ }
+
+ str = pnstrdup(value->val.string.val, value->val.string.len);
+ if (length)
+ *length = value->val.string.len;
+ pfree(value);
+ return str;
+}
+
+#define ADVANCE_PARSE_POINTER(ptr,end_ptr) \
+ do { \
+ if (++(ptr) >= (end_ptr)) \
+ ereport(ERROR, \
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE), \
+ errmsg("unterminated format specifier"))); \
+ } while (0)
+
+/*
+ * Recursive helper for pg_event_trigger_expand_command
+ *
+ * Find the "fmt" element in the given container, and expand it into the
+ * provided StringInfo.
+ */
+static void
+expand_fmt_recursive(JsonbContainer *container, StringInfo buf)
+{
+ JsonbValue key;
+ JsonbValue *value;
+ const char *cp;
+ const char *start_ptr;
+ const char *end_ptr;
+ int len;
+
+ start_ptr = find_string_in_jsonbcontainer(container, "fmt", false, &len);
+ end_ptr = start_ptr + len;
+
+ for (cp = start_ptr; cp < end_ptr; cp++)
+ {
+ convSpecifier specifier;
+ bool is_array;
+ char *param = NULL;
+ char *arraysep = NULL;
+
+ if (*cp != '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+
+ /* Easy case: %% outputs a single % */
+ if (*cp == '%')
+ {
+ appendStringInfoCharMacro(buf, *cp);
+ continue;
+ }
+
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
+ if (*cp == '{')
+ {
+ StringInfoData parbuf;
+ StringInfoData arraysepbuf;
+ StringInfo appendTo;
+
+ initStringInfo(&parbuf);
+ appendTo = &parbuf;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ for (; cp < end_ptr;)
+ {
+ if (*cp == ':')
+ {
+ /*
+ * Found array separator delimiter; element name is now
+ * complete, start filling the separator.
+ */
+ initStringInfo(&arraysepbuf);
+ appendTo = &arraysepbuf;
+ is_array = true;
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ continue;
+ }
+
+ if (*cp == '}')
+ {
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ break;
+ }
+ appendStringInfoCharMacro(appendTo, *cp);
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
+ }
+ param = parbuf.data;
+ if (is_array)
+ arraysep = arraysepbuf.data;
+ }
+ if (param == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing conversion name in conversion specifier")));
+
+ switch (*cp)
+ {
+ case 'I':
+ specifier = SpecIdentifier;
+ break;
+ case 'D':
+ specifier = SpecDottedName;
+ break;
+ case 's':
+ specifier = SpecString;
+ break;
+ case 'L':
+ specifier = SpecStringLiteral;
+ break;
+ case 'T':
+ specifier = SpecTypename;
+ break;
+ case 'O':
+ specifier = SpecOperatorname;
+ break;
+ case 'n':
+ specifier = SpecNumber;
+ break;
+ case 'R':
+ specifier = SpecRole;
+ break;
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("invalid conversion specifier \"%c\"", *cp)));
+ }
+
+ /*
+ * Obtain the element to be expanded.
+ */
+ key.type = jbvString;
+ key.val.string.val = param;
+ key.val.string.len = strlen(param);
+
+ value = findJsonbValueFromContainer(container, JB_FOBJECT, &key);
+
+ /*
+ * Validate that we got an array if the format string specified one.
+ * And finally print out the data
+ */
+ if (is_array)
+ expand_jsonb_array(buf, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(buf, param, value, specifier, start_ptr);
+ }
+}
+
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ Assert(jsonval->type == jbvString);
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+ appendStringInfoString(buf, quote_identifier(str));
+ pfree(str);
+}
+
+/*
+ * Expand a json value as a dot-separated-name. The value must be of type
+ * object and may contain elements "schemaname" (optional), "objname"
+ * (mandatory), "attrname" (optional). Double quotes are added to each element
+ * as necessary, and dot separators where needed.
+ *
+ * One day we might need a "catalog" element as well, but no current use case
+ * needs that.
+ */
+static void
+expand_jsonval_dottedname(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+ pfree(str);
+ }
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfo(buf, "%s", quote_identifier(str));
+ pfree(str);
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "attrname", true, NULL);
+ if (str)
+ {
+ appendStringInfo(buf, ".%s", quote_identifier(str));
+ pfree(str);
+ }
+}
+
+/*
+ * Expand a json value as a type name.
+ */
+static void
+expand_jsonval_typename(StringInfo buf, JsonbValue *jsonval)
+{
+ char *schema = NULL;
+ char *typename;
+ char *typmodstr;
+ trivalue is_array;
+ char *array_decor;
+
+ /*
+ * We omit schema-qualifying the output name if the schema element is
+ * either the empty string or NULL; the difference between those two cases
+ * is that in the latter we quote the type name, in the former we don't.
+ * This allows for types with special typmod needs, such as interval and
+ * timestamp (see format_type_detailed), while at the same time allowing
+ * for the schema name to be omitted from type names that require quotes
+ * but are to be obtained from a user schema.
+ */
+
+ schema = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ typename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typename", false, NULL);
+ typmodstr = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "typmod", true, NULL);
+ is_array = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "typarray");
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("missing typarray element")));
+ break;
+ case tv_true:
+ array_decor = "[]";
+ break;
+ case tv_false:
+ array_decor = "";
+ break;
+ }
+
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+}
+
+/*
+ * Expand a json value as an operator name
+ */
+static void
+expand_jsonval_operator(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "schemaname", true, NULL);
+ /* schema might be NULL or empty */
+ if (str != NULL && str[0] != '\0')
+ appendStringInfo(buf, "%s.", quote_identifier(str));
+
+ str = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "objname", false, NULL);
+ appendStringInfoString(buf, str);
+}
+
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
+static bool
+expand_jsonval_string(StringInfo buf, JsonbValue *jsonval)
+{
+ if (jsonval->type == jbvString)
+ {
+ appendBinaryStringInfo(buf, jsonval->val.string.val,
+ jsonval->val.string.len);
+ }
+ else if (jsonval->type == jbvBinary)
+ {
+ trivalue present;
+
+ present = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "present");
+ /*
+ * If "present" is set to false, this element expands to empty;
+ * otherwise (either true or absent), fall through to expand "fmt".
+ */
+ if (present == tv_false)
+ return false;
+
+ expand_fmt_recursive(jsonval->val.binary.data, buf);
+ }
+ return true;
+}
+
+/*
+ * Expand a json value as a string literal.
+ */
+static void
+expand_jsonval_strlit(StringInfo buf, JsonbValue *jsonval)
+{
+ char *str;
+ StringInfoData dqdelim;
+ static const char dqsuffixes[] = "_XYZZYX_";
+ int dqnextchar = 0;
+
+ str = pnstrdup(jsonval->val.string.val, jsonval->val.string.len);
+
+ /* Easy case: if there are no ' and no \, just use a single quote */
+ if (strchr(str, '\'') == NULL &&
+ strchr(str, '\\') == NULL)
+ {
+ appendStringInfo(buf, "'%s'", str);
+ pfree(str);
+ return;
+ }
+
+ /* Otherwise need to find a useful dollar-quote delimiter */
+ initStringInfo(&dqdelim);
+ appendStringInfoString(&dqdelim, "$");
+ while (strstr(str, dqdelim.data) != NULL)
+ {
+ appendStringInfoChar(&dqdelim, dqsuffixes[dqnextchar++]);
+ dqnextchar = dqnextchar % (sizeof(dqsuffixes) - 1);
+ }
+ /* Add trailing $ */
+ appendStringInfoChar(&dqdelim, '$');
+
+ /* And finally produce the quoted literal into the output StringInfo */
+ appendStringInfo(buf, "%s%s%s", dqdelim.data, str, dqdelim.data);
+ pfree(dqdelim.data);
+ pfree(str);
+}
+
+/*
+ * Expand a json value as an integer quantity.
+ */
+static void
+expand_jsonval_number(StringInfo buf, JsonbValue *jsonval)
+{
+ char *strdatum;
+
+ strdatum = DatumGetCString(DirectFunctionCall1(numeric_out,
+ NumericGetDatum(jsonval->val.numeric)));
+ appendStringInfoString(buf, strdatum);
+}
+
+/*
+ * Expand a json value as a role name. If the is_public element is set to
+ * true, PUBLIC is expanded (no quotes); otherwise, expand the given role name,
+ * quoting as an identifier.
+ */
+static void
+expand_jsonval_role(StringInfo buf, JsonbValue *jsonval)
+{
+ trivalue is_public;
+
+ is_public = find_bool_in_jsonbcontainer(jsonval->val.binary.data,
+ "is_public");
+ if (is_public == tv_true)
+ appendStringInfoString(buf, "PUBLIC");
+ else
+ {
+ char *rolename;
+
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
+ }
+}
+
+/*
+ * Expand one json element into the output StringInfo according to the
+ * conversion specifier. The element type is validated, and an error is raised
+ * if it doesn't match what we expect for the conversion specifier.
+ *
+ * Returns false if no actual expansion was made (due to the "present" flag
+ * being set to "false" in formatted string expansion).
+ */
+static bool
+expand_one_jsonb_element(StringInfo buf, char *param, JsonbValue *jsonval,
+ convSpecifier specifier, const char *fmt)
+{
+ bool result = true;
+ ErrorContextCallback sqlerrcontext;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (!jsonval)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" not found", param)));
+
+ switch (specifier)
+ {
+ case SpecIdentifier:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%I element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_identifier(buf, jsonval);
+ break;
+
+ case SpecDottedName:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%D element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_dottedname(buf, jsonval);
+ break;
+
+ case SpecString:
+ if (jsonval->type != jbvString &&
+ jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string or object for %%s element \"%s\", got %d",
+ param, jsonval->type)));
+ result = expand_jsonval_string(buf, jsonval);
+ break;
+
+ case SpecStringLiteral:
+ if (jsonval->type != jbvString)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON string for %%L element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_strlit(buf, jsonval);
+ break;
+
+ case SpecTypename:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%T element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_typename(buf, jsonval);
+ break;
+
+ case SpecOperatorname:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%O element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_operator(buf, jsonval);
+ break;
+
+ case SpecNumber:
+ if (jsonval->type != jbvNumeric)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON numeric for %%n element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_number(buf, jsonval);
+ break;
+
+ case SpecRole:
+ if (jsonval->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("expected JSON object for %%R element \"%s\", got %d",
+ param, jsonval->type)));
+ expand_jsonval_role(buf, jsonval);
+ break;
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+
+ return result;
+}
+
+/*
+ * Iterate on the elements of a JSON array, expanding each one into the output
+ * StringInfo per the given conversion specifier, separated by the given
+ * separator.
+ */
+static void
+expand_jsonb_array(StringInfo buf, char *param,
+ JsonbValue *jsonarr, char *arraysep, convSpecifier specifier,
+ const char *fmt)
+{
+ ErrorContextCallback sqlerrcontext;
+ JsonbContainer *container;
+ JsonbIterator *it;
+ JsonbValue v;
+ int type;
+ bool first = true;
+ StringInfoData arrayelem;
+
+ /* If we were given a format string, setup an ereport() context callback */
+ if (fmt)
+ {
+ sqlerrcontext.callback = fmtstr_error_callback;
+ sqlerrcontext.arg = (void *) fmt;
+ sqlerrcontext.previous = error_context_stack;
+ error_context_stack = &sqlerrcontext;
+ }
+
+ if (jsonarr->type != jbvBinary)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ container = jsonarr->val.binary.data;
+ if (!JsonContainerIsArray(container))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("element \"%s\" is not a JSON array", param)));
+
+ initStringInfo(&arrayelem);
+
+ it = JsonbIteratorInit(container);
+ while ((type = JsonbIteratorNext(&it, &v, true)) != WJB_DONE)
+ {
+ if (type == WJB_ELEM)
+ {
+ resetStringInfo(&arrayelem);
+
+ if (expand_one_jsonb_element(&arrayelem, param, &v, specifier, NULL))
+ {
+ if (!first)
+ appendStringInfoString(buf, arraysep);
+
+ appendBinaryStringInfo(buf, arrayelem.data, arrayelem.len);
+ first = false;
+ }
+ }
+ }
+
+ if (fmt)
+ error_context_stack = sqlerrcontext.previous;
+}
+
+/*
+ * Workhorse for ddl_deparse_expand_command.
+ */
+char *
+ddl_deparse_json_to_string(char *json_str)
+{
+ Datum d;
+ Jsonb *jsonb;
+ StringInfo buf = (StringInfo) palloc0(sizeof(StringInfoData));
+
+ initStringInfo(buf);
+
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
+ jsonb = (Jsonb *) DatumGetPointer(d);
+
+ expand_fmt_recursive(&jsonb->root, buf);
+
+ return buf->data;
+}
+
+/*------
+ * Returns a formatted string from a JSON object.
+ *
+ * The starting point is the element named "fmt" (which must be a string).
+ * This format string may contain zero or more %-escapes, which consist of an
+ * element name enclosed in { }, possibly followed by a conversion modifier,
+ * followed by a conversion specifier. Possible conversion specifiers are:
+ *
+ * % expand to a literal %.
+ * I expand as a single, non-qualified identifier
+ * D expand as a possibly-qualified identifier
+ * T expand as a type name
+ * O expand as an operator name
+ * L expand as a string literal (quote using single quotes)
+ * s expand as a simple string (no quoting)
+ * n expand as a simple number (no quoting)
+ * R expand as a role name (possibly quoted name, or PUBLIC)
+ *
+ * The element name may have an optional separator specification preceded
+ * by a colon. Its presence indicates that the element is expected to be
+ * an array; the specified separator is used to join the array elements.
+ *------
+ */
+Datum
+ddl_deparse_expand_command(PG_FUNCTION_ARGS)
+{
+ text *json = PG_GETARG_TEXT_P(0);
+ char *json_str;
+
+ json_str = TextDatumGetCString(json);
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(ddl_deparse_json_to_string(json_str)));
+}
+
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable.
+ */
+static void
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index b0b2118..289a5de 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1700,6 +1700,39 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
relation_close(tablerel, NoLock);
}
+/*
+ * Return sequence parameters, detailed
+ */
+Form_pg_sequence_data
+get_sequence_values(Oid sequenceId)
+{
+ Buffer buf;
+ SeqTable elm;
+ Relation seqrel;
+ HeapTupleData seqtuple;
+ Form_pg_sequence_data seq;
+ Form_pg_sequence_data retSeq;
+
+ /* Open and AccessShareLock sequence */
+ init_sequence(sequenceId, &elm, &seqrel);
+
+ if (pg_class_aclcheck(sequenceId, GetUserId(),
+ ACL_SELECT | ACL_UPDATE | ACL_USAGE) != ACLCHECK_OK)
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied for sequence %s",
+ RelationGetRelationName(seqrel))));
+
+ seq = read_seq_tuple(seqrel, &buf, &seqtuple);
+ retSeq = palloc(sizeof(FormData_pg_sequence_data));
+
+ memcpy(retSeq, seq, sizeof(FormData_pg_sequence_data));
+
+ UnlockReleaseBuffer(buf);
+ relation_close(seqrel, NoLock);
+
+ return retSeq;
+}
/*
* Return sequence parameters in a list of the form created by the parser.
diff --git a/src/backend/utils/adt/format_type.c b/src/backend/utils/adt/format_type.c
index 060fd7e..6af2257 100644
--- a/src/backend/utils/adt/format_type.c
+++ b/src/backend/utils/adt/format_type.c
@@ -27,7 +27,6 @@
#include "utils/numeric.h"
#include "utils/syscache.h"
-static char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
/*
@@ -367,7 +366,7 @@ format_type_with_typemod(Oid type_oid, int32 typemod)
/*
* Add typmod decoration to the basic type name
*/
-static char *
+char *
printTypmod(const char *typname, int32 typmod, Oid typmodout)
{
char *res;
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index d575aa0..95e6780 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -358,7 +358,6 @@ static int print_function_arguments(StringInfo buf, HeapTuple proctup,
bool print_table_args, bool print_defaults);
static void print_function_rettype(StringInfo buf, HeapTuple proctup);
static void print_function_trftypes(StringInfo buf, HeapTuple proctup);
-static void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
static void set_rtable_names(deparse_namespace *dpns, List *parent_namespaces,
Bitmapset *rels_used);
static void set_deparse_for_query(deparse_namespace *dpns, Query *query,
@@ -488,22 +487,15 @@ static void get_from_clause_coldeflist(RangeTblFunction *rtfunc,
deparse_context *context);
static void get_tablesample_def(TableSampleClause *tablesample,
deparse_context *context);
-static void get_opclass_name(Oid opclass, Oid actual_datatype,
- StringInfo buf);
static Node *processIndirection(Node *node, deparse_context *context);
static void printSubscripts(SubscriptingRef *sbsref, deparse_context *context);
static char *get_relation_name(Oid relid);
static char *generate_relation_name(Oid relid, List *namespaces);
static char *generate_qualified_relation_name(Oid relid);
-static char *generate_function_name(Oid funcid, int nargs,
- List *argnames, Oid *argtypes,
- bool has_variadic, bool *use_variadic_p,
- ParseExprKind special_exprkind);
static char *generate_operator_name(Oid operid, Oid arg1, Oid arg2);
static void add_cast_to(StringInfo buf, Oid typid);
static char *generate_qualified_type_name(Oid typid);
static text *string_to_text(char *str);
-static char *flatten_reloptions(Oid relid);
static void get_reloptions(StringInfo buf, Datum reloptions);
static void get_json_path_spec(Node *path_spec, deparse_context *context,
bool showimplicit);
@@ -1025,65 +1017,12 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
if (!isnull)
{
Node *qual;
- char relkind;
- deparse_context context;
- deparse_namespace dpns;
- RangeTblEntry *oldrte;
- RangeTblEntry *newrte;
-
- appendStringInfoString(&buf, "WHEN (");
+ char *qualstr;
qual = stringToNode(TextDatumGetCString(value));
+ qualstr = pg_get_trigger_whenclause(trigrec, qual, pretty);
- relkind = get_rel_relkind(trigrec->tgrelid);
-
- /* Build minimal OLD and NEW RTEs for the rel */
- oldrte = makeNode(RangeTblEntry);
- oldrte->rtekind = RTE_RELATION;
- oldrte->relid = trigrec->tgrelid;
- oldrte->relkind = relkind;
- oldrte->rellockmode = AccessShareLock;
- oldrte->alias = makeAlias("old", NIL);
- oldrte->eref = oldrte->alias;
- oldrte->lateral = false;
- oldrte->inh = false;
- oldrte->inFromCl = true;
-
- newrte = makeNode(RangeTblEntry);
- newrte->rtekind = RTE_RELATION;
- newrte->relid = trigrec->tgrelid;
- newrte->relkind = relkind;
- newrte->rellockmode = AccessShareLock;
- newrte->alias = makeAlias("new", NIL);
- newrte->eref = newrte->alias;
- newrte->lateral = false;
- newrte->inh = false;
- newrte->inFromCl = true;
-
- /* Build two-element rtable */
- memset(&dpns, 0, sizeof(dpns));
- dpns.rtable = list_make2(oldrte, newrte);
- dpns.subplans = NIL;
- dpns.ctes = NIL;
- dpns.appendrels = NULL;
- set_rtable_names(&dpns, NIL, NULL);
- set_simple_column_names(&dpns);
-
- /* Set up context with one-deep namespace stack */
- context.buf = &buf;
- context.namespaces = list_make1(&dpns);
- context.windowClause = NIL;
- context.windowTList = NIL;
- context.varprefix = true;
- context.prettyFlags = GET_PRETTY_FLAGS(pretty);
- context.wrapColumn = WRAP_COLUMN_DEFAULT;
- context.indentLevel = PRETTYINDENT_STD;
- context.special_exprkind = EXPR_KIND_NONE;
- context.appendparents = NULL;
-
- get_rule_expr(qual, &context, false);
-
- appendStringInfoString(&buf, ") ");
+ appendStringInfo(&buf, "WHEN (%s) ", qualstr);
}
appendStringInfo(&buf, "EXECUTE FUNCTION %s(",
@@ -1124,6 +1063,64 @@ pg_get_triggerdef_worker(Oid trigid, bool pretty)
return buf.data;
}
+char *
+pg_get_trigger_whenclause(Form_pg_trigger trigrec, Node *whenClause, bool pretty)
+{
+ StringInfoData buf;
+ char relkind;
+ deparse_context context;
+ deparse_namespace dpns;
+ RangeTblEntry *oldrte;
+ RangeTblEntry *newrte;
+
+ initStringInfo(&buf);
+
+ relkind = get_rel_relkind(trigrec->tgrelid);
+
+ /* Build minimal OLD and NEW RTEs for the rel */
+ oldrte = makeNode(RangeTblEntry);
+ oldrte->rtekind = RTE_RELATION;
+ oldrte->relid = trigrec->tgrelid;
+ oldrte->relkind = relkind;
+ oldrte->alias = makeAlias("old", NIL);
+ oldrte->eref = oldrte->alias;
+ oldrte->lateral = false;
+ oldrte->inh = false;
+ oldrte->inFromCl = true;
+
+ newrte = makeNode(RangeTblEntry);
+ newrte->rtekind = RTE_RELATION;
+ newrte->relid = trigrec->tgrelid;
+ newrte->relkind = relkind;
+ newrte->alias = makeAlias("new", NIL);
+ newrte->eref = newrte->alias;
+ newrte->lateral = false;
+ newrte->inh = false;
+ newrte->inFromCl = true;
+
+ /* Build two-element rtable */
+ memset(&dpns, 0, sizeof(dpns));
+ dpns.rtable = list_make2(oldrte, newrte);
+ dpns.ctes = NIL;
+ set_rtable_names(&dpns, NIL, NULL);
+ set_simple_column_names(&dpns);
+
+ /* Set up context with one-deep namespace stack */
+ context.buf = &buf;
+ context.namespaces = list_make1(&dpns);
+ context.windowClause = NIL;
+ context.windowTList = NIL;
+ context.varprefix = true;
+ context.prettyFlags = pretty ? PRETTYFLAG_PAREN | PRETTYFLAG_INDENT : PRETTYFLAG_INDENT;
+ context.wrapColumn = WRAP_COLUMN_DEFAULT;
+ context.indentLevel = PRETTYINDENT_STD;
+ context.special_exprkind = EXPR_KIND_NONE;
+
+ get_rule_expr(whenClause, &context, false);
+
+ return buf.data;
+}
+
/* ----------
* pg_get_indexdef - Get the definition of an index
*
@@ -1891,6 +1888,13 @@ pg_get_partkeydef_columns(Oid relid, bool pretty)
return pg_get_partkeydef_worker(relid, prettyFlags, true, false);
}
+/* Internal version that reports the full partition key definition */
+char *
+pg_get_partkeydef_simple(Oid relid)
+{
+ return pg_get_partkeydef_worker(relid, 0, false, false);
+}
+
/*
* Internal workhorse to decompile a partition key definition.
*/
@@ -2144,6 +2148,15 @@ pg_get_constraintdef_ext(PG_FUNCTION_ARGS)
}
/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
+char *
+pg_get_constraintdef_command_simple(Oid constraintId)
+{
+ return pg_get_constraintdef_worker(constraintId, false, 0, false);
+}
+
+/*
* Internal version that returns a full ALTER TABLE ... ADD CONSTRAINT command
*/
char *
@@ -3513,7 +3526,7 @@ pg_get_function_arg_default(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(string_to_text(str));
}
-static void
+void
print_function_sqlbody(StringInfo buf, HeapTuple proctup)
{
int numargs;
@@ -12053,7 +12066,7 @@ get_tablesample_def(TableSampleClause *tablesample, deparse_context *context)
* actual_datatype. (If you don't want this behavior, just pass
* InvalidOid for actual_datatype.)
*/
-static void
+void
get_opclass_name(Oid opclass, Oid actual_datatype,
StringInfo buf)
{
@@ -12447,7 +12460,7 @@ generate_qualified_relation_name(Oid relid)
*
* The result includes all necessary quoting and schema-prefixing.
*/
-static char *
+char *
generate_function_name(Oid funcid, int nargs, List *argnames, Oid *argtypes,
bool has_variadic, bool *use_variadic_p,
ParseExprKind special_exprkind)
@@ -12833,7 +12846,7 @@ get_reloptions(StringInfo buf, Datum reloptions)
/*
* Generate a C string representing a relation's reloptions, or NULL if none.
*/
-static char *
+char *
flatten_reloptions(Oid relid)
{
char *result = NULL;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be47583..3e7f364 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11894,4 +11894,10 @@
prorettype => 'bytea', proargtypes => 'pg_brin_minmax_multi_summary',
prosrc => 'brin_minmax_multi_summary_send' },
+{ oid => '4642', descr => 'deparse the DDL command into json format string',
+ proname => 'ddl_deparse_to_json', prorettype => 'text',
+ proargtypes => 'pg_ddl_command', prosrc => 'ddl_deparse_to_json' },
+{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
+ proname => 'ddl_deparse_expand_command', prorettype => 'text',
+ proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
]
diff --git a/src/include/commands/sequence.h b/src/include/commands/sequence.h
index d38c0e2..4ca84b2 100644
--- a/src/include/commands/sequence.h
+++ b/src/include/commands/sequence.h
@@ -54,6 +54,7 @@ typedef struct xl_seq_rec
extern int64 nextval_internal(Oid relid, bool check_permissions);
extern Datum nextval(PG_FUNCTION_ARGS);
extern List *sequence_options(Oid relid);
+extern Form_pg_sequence_data get_sequence_values(Oid sequenceId);
extern ObjectAddress DefineSequence(ParseState *pstate, CreateSeqStmt *stmt);
extern ObjectAddress AlterSequence(ParseState *pstate, AlterSeqStmt *stmt);
diff --git a/src/include/tcop/ddl_deparse.h b/src/include/tcop/ddl_deparse.h
new file mode 100644
index 0000000..bdfdf47
--- /dev/null
+++ b/src/include/tcop/ddl_deparse.h
@@ -0,0 +1,23 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddl_deparse.h
+ *
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/tcop/ddl_deparse.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef DDL_DEPARSE_H
+#define DDL_DEPARSE_H
+
+#include "tcop/deparse_utility.h"
+
+extern char *deparse_utility_command(CollectedCommand *cmd, bool verbose_mode);
+extern char *ddl_deparse_json_to_string(char *jsonb);
+extern char *deparse_drop_command(const char *objidentity, const char *objecttype,
+ DropBehavior behavior);
+
+
+#endif /* DDL_DEPARSE_H */
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index 94de13d..b53294b 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -62,6 +62,7 @@ typedef struct CollectedCommand
{
Oid objectId;
Oid classId;
+ bool rewrite;
List *subcmds;
} alterTable;
diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h
index 221c3e6..85a62a1 100644
--- a/src/include/utils/builtins.h
+++ b/src/include/utils/builtins.h
@@ -118,6 +118,7 @@ extern char *format_type_extended(Oid type_oid, int32 typemod, bits16 flags);
extern char *format_type_be(Oid type_oid);
extern char *format_type_be_qualified(Oid type_oid);
extern char *format_type_with_typemod(Oid type_oid, int32 typemod);
+extern char *printTypmod(const char *typname, int32 typmod, Oid typmodout);
extern int32 type_maximum_size(Oid type_oid, int32 typemod);
diff --git a/src/include/utils/ruleutils.h b/src/include/utils/ruleutils.h
index 7d48971..3d69a60 100644
--- a/src/include/utils/ruleutils.h
+++ b/src/include/utils/ruleutils.h
@@ -13,6 +13,7 @@
#ifndef RULEUTILS_H
#define RULEUTILS_H
+#include "catalog/pg_trigger.h"
#include "nodes/nodes.h"
#include "nodes/parsenodes.h"
#include "nodes/pg_list.h"
@@ -23,12 +24,16 @@ struct PlannedStmt;
extern char *pg_get_indexdef_string(Oid indexrelid);
extern char *pg_get_indexdef_columns(Oid indexrelid, bool pretty);
+extern char *pg_get_trigger_whenclause(Form_pg_trigger trigrec,
+ Node *whenClause, bool pretty);
extern char *pg_get_querydef(Query *query, bool pretty);
extern char *pg_get_partkeydef_columns(Oid relid, bool pretty);
+extern char *pg_get_partkeydef_simple(Oid relid);
extern char *pg_get_partconstrdef_string(Oid partitionId, char *aliasname);
extern char *pg_get_constraintdef_command(Oid constraintId);
+extern char *pg_get_constraintdef_command_simple(Oid constraintId);
extern char *deparse_expression(Node *expr, List *dpcontext,
bool forceprefix, bool showimplicit);
extern List *deparse_context_for(const char *aliasname, Oid relid);
@@ -40,8 +45,16 @@ extern List *select_rtable_names_for_explain(List *rtable,
Bitmapset *rels_used);
extern char *generate_collation_name(Oid collid);
extern char *generate_opclass_name(Oid opclass);
+extern char *generate_function_name(Oid funcid, int nargs, List *argnames,
+ Oid *argtypes, bool has_variadic,
+ bool *use_variadic_p,
+ ParseExprKind special_exprkind);
extern char *get_range_partbound_string(List *bound_datums);
+extern void get_opclass_name(Oid opclass, Oid actual_datatype,
+ StringInfo buf);
+extern char *flatten_reloptions(Oid relid);
extern char *pg_get_statisticsobjdef_string(Oid statextid);
+extern void print_function_sqlbody(StringInfo buf, HeapTuple proctup);
#endif /* RULEUTILS_H */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 35c9f1e..cd9dad1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1621,6 +1621,9 @@ OSInfo
OSSLCipher
OSSLDigest
OVERLAPPED
+ObjElem
+ObjTree
+ObjType
ObjectAccessDrop
ObjectAccessNamespaceSearch
ObjectAccessPostAlter
@@ -3188,6 +3191,7 @@ colormaprange
compare_context
config_var_value
contain_aggs_of_level_context
+convSpecifier
convert_testexpr_context
copy_data_source_cb
core_YYSTYPE
--
2.7.2.windows.1
[application/octet-stream] v15-0002-Support-DDL-replication.patch (123.8K, ../../OS0PR01MB5716009FDCCC0B50BCB14A99949D9@OS0PR01MB5716.jpnprd01.prod.outlook.com/4-v15-0002-Support-DDL-replication.patch)
download | inline diff:
From 97e369bb748d512b8321ee3c6836d4dfede6b9fa Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 28 Jul 2022 10:29:23 +0800
Subject: [PATCH v15 2/4] Support DDL replication.
To support DDL replication, we use event trigger and DDL deparsing
facilities. While creating a publication, we register a command end
trigger that deparses the DDL as a JSON blob, and WAL logs it. The event
trigger is automatically removed at the time of drop publication. The
WALSender decodes the WAL and sends it downstream similar to other DML
commands. The subscriber then converts JSON back to the DDL command string
and executes it. In the subscriber, we also add the newly added rel to
pg_subscription_rel so that the DML changes on the new table can be
replicated without having to manually run
"ALTER SUBSCRIPTION ... REFRESH PUBLICATION".
This is a POC patch to show how using event triggers and DDL deparsing
facilities we can implement DDL replication. So, the implementation is
restricted to CREATE TABLE/ALTER TABLE/DROP TABLE commands.
- For non-rewrite ALTER [TABLE|SEQUENCE|SCHEMA] command and
- CREATE [TABLE|SEQUENCE|SCHEMA|INDEX] command:
- And other table unrelated command
we deparse the command at ddl_command_end event trigger and WAL log the
deparsed json string. The WALSender decodes the WAL and sends it to
subscriber if the created/altered table is published. It supports most of
ALTER TABLE command except some commands(DDL related to PARTITIONED TABLE
...) that introduced recently which haven't been supported by the current
ddl_deparser, we will support that later.
Note that the replication for ALTER INDEX command is still under
progress.
- For DROP [TABLE|SEQUENCE|SCHEMA|INDEX]:
The 'command start' event handler logs a ddl message with the relids of
the tables that are dropped which the output plugin (pgoutput) stores in
its internal data structure after verifying that it is for a table that is
part of the publication. Later the 'command end' event handler sends the
actual drop message. Pgoutput on receiving the command end, only sends out
the drop command only if it is for one of the relids marked for deleting.
The reason we have to do this is because, once the logical decoder
receives the 'command end' message, the relid of the table is no longer
valid as it has been deleted as part of invalidations received for the
drop table command. It is no longer possible to verify if the table is
part of the publication list or not. To make this possible, I have added
two more elements to the ddl xlog and ddl message, (relid and cmdtype).
We could have also handled all this on the subscriber side as well, but
that would mean sending spurious ddl messages for tables that are not part
of the publication.
- For table_rewrite ALTER TABLE command:
(ALTER COLUMN TYPE, ADD COLUMN DEFAULT, SET LOGGED, SET ACCESS METHOD)
we deparse the command and WAL log the deparsed json string at
table_rewrite event trigger. The WALSender decodes the WAL and sends it to
subscriber if the altered table is published. Then, the WALSender will
convert the upcoming rewrite INSERTs to UPDATEs and send them to
subscriber so that the data between publisher and subscriber can always be
consistent. Note that the tables that publish rewrite ddl must have a
replica identity configured in order to be able to replicate the upcoming
rewrite UPDATEs.
We do this way because of two reason:
(1) The data before the rewrite ddl could already be different among
publisher and subscriber. To make sure the extra data in subscriber which
doesn't exist in publisher also get rewritten, we need to let the
subscriber execute the original rewrite ddl to rewrite all the data at
first.
(2) the data after executing rewrite ddl could be different among
publisher and subscriber(due to different functions/operators used during
rewrite), so we need to replicate the rewrite UPDATEs to keep the data
consistent.
TO IMPROVE:
This approach could be improved by letting the subscriber try to update
the extra data itself instead of doing fully rewrite ddl and use the
upcoming rewrite UPDATEs to rewrite the rest data. To achieve this, we
could modify the deparsed json string to temporarily remove the rewrite
part and add some logic in subscriber to update the extra data.
Besides, we may not need to send rewrite changes for all type of rewrite
ddl, for example, it seems fine to skip sending rewrite changes for ALTER
TABLE SET LOGGED as the data in the table doesn't actually be changed. We
could use the deparser and event trigger to filter these ddls and skip
sending rewrite changes for them.
---
src/backend/access/rmgrdesc/Makefile | 1 +
src/backend/access/rmgrdesc/logicalddlmsgdesc.c | 52 ++++
src/backend/catalog/pg_publication.c | 1 +
src/backend/commands/event_trigger.c | 214 ++++++++++++-
src/backend/commands/publicationcmds.c | 112 +++++++
src/backend/commands/tablecmds.c | 2 +-
src/backend/replication/logical/Makefile | 1 +
src/backend/replication/logical/ddlmessage.c | 86 ++++++
src/backend/replication/logical/decode.c | 41 +++
src/backend/replication/logical/logical.c | 93 ++++++
src/backend/replication/logical/proto.c | 48 +++
src/backend/replication/logical/reorderbuffer.c | 136 +++++++++
src/backend/replication/logical/worker.c | 232 ++++++++++++++
src/backend/replication/pgoutput/pgoutput.c | 192 +++++++++++-
src/backend/utils/cache/relcache.c | 1 +
src/bin/pg_dump/pg_dump.c | 27 +-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_waldump/rmgrdesc.c | 1 +
src/bin/psql/describe.c | 17 +-
src/include/access/rmgrlist.h | 1 +
src/include/catalog/pg_proc.dat | 9 +
src/include/catalog/pg_publication.h | 4 +
src/include/commands/event_trigger.h | 3 +-
src/include/replication/ddlmessage.h | 60 ++++
src/include/replication/decode.h | 1 +
src/include/replication/logicalproto.h | 7 +-
src/include/replication/output_plugin.h | 27 ++
src/include/replication/pgoutput.h | 1 +
src/include/replication/reorderbuffer.h | 39 +++
src/test/regress/expected/psql.out | 6 +-
src/test/regress/expected/publication.out | 388 ++++++++++++------------
31 files changed, 1591 insertions(+), 213 deletions(-)
create mode 100644 src/backend/access/rmgrdesc/logicalddlmsgdesc.c
create mode 100644 src/backend/replication/logical/ddlmessage.c
create mode 100644 src/include/replication/ddlmessage.h
diff --git a/src/backend/access/rmgrdesc/Makefile b/src/backend/access/rmgrdesc/Makefile
index f88d72f..b8e29e8 100644
--- a/src/backend/access/rmgrdesc/Makefile
+++ b/src/backend/access/rmgrdesc/Makefile
@@ -19,6 +19,7 @@ OBJS = \
hashdesc.o \
heapdesc.o \
logicalmsgdesc.o \
+ logicalddlmsgdesc.o \
mxactdesc.o \
nbtdesc.o \
relmapdesc.o \
diff --git a/src/backend/access/rmgrdesc/logicalddlmsgdesc.c b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
new file mode 100644
index 0000000..81dee52
--- /dev/null
+++ b/src/backend/access/rmgrdesc/logicalddlmsgdesc.c
@@ -0,0 +1,52 @@
+/*-------------------------------------------------------------------------
+ *
+ * logicalddlmsgdesc.c
+ * rmgr descriptor routines for replication/logical/ddlmessage.c
+ *
+ * Portions Copyright (c) 2015-2022, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/rmgrdesc/logicalddlmsgdesc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "replication/ddlmessage.h"
+
+void
+logicalddlmsg_desc(StringInfo buf, XLogReaderState *record)
+{
+ char *rec = XLogRecGetData(record);
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info == XLOG_LOGICAL_DDL_MESSAGE)
+ {
+ xl_logical_ddl_message *xlrec = (xl_logical_ddl_message *) rec;
+ char *prefix = xlrec->message;
+ char *message = xlrec->message + xlrec->prefix_size;
+ char *sep = "";
+
+ Assert(prefix[xlrec->prefix_size] != '\0');
+
+ appendStringInfo(buf, "prefix \"%s\"; payload (%zu bytes): ",
+ prefix, xlrec->message_size);
+ appendStringInfo(buf, "relid %u cmdtype %u", xlrec->relid, xlrec->cmdtype);
+ /* Write message payload as a series of hex bytes */
+ for (int cnt = 0; cnt < xlrec->message_size; cnt++)
+ {
+ appendStringInfo(buf, "%s%02X", sep, (unsigned char) message[cnt]);
+ sep = " ";
+ }
+ }
+}
+
+const char *
+logicalddlmsg_identify(uint8 info)
+{
+ if ((info & ~XLR_INFO_MASK) == XLOG_LOGICAL_DDL_MESSAGE)
+ return "DDL MESSAGE";
+
+ return NULL;
+}
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index e294bea..4439b63 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -1005,6 +1005,7 @@ GetPublication(Oid pubid)
pub->pubactions.pubupdate = pubform->pubupdate;
pub->pubactions.pubdelete = pubform->pubdelete;
pub->pubactions.pubtruncate = pubform->pubtruncate;
+ pub->pubactions.pubddl = pubform->pubddl;
pub->pubviaroot = pubform->pubviaroot;
ReleaseSysCache(tup);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index eef3e5d..a00f061 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -37,8 +37,11 @@
#include "miscadmin.h"
#include "parser/parse_func.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
+#include "replication/message.h"
#include "tcop/deparse_utility.h"
#include "tcop/utility.h"
+#include "tcop/ddl_deparse.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/evtcache.h"
@@ -1536,6 +1539,7 @@ EventTriggerAlterTableStart(Node *parsetree)
command->d.alterTable.classId = RelationRelationId;
command->d.alterTable.objectId = InvalidOid;
+ command->d.alterTable.rewrite = false;
command->d.alterTable.subcmds = NIL;
command->parsetree = copyObject(parsetree);
@@ -1569,7 +1573,7 @@ EventTriggerAlterTableRelid(Oid objectId)
* internally, so that's all that this code needs to handle at the moment.
*/
void
-EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
+EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address, bool rewrite)
{
MemoryContext oldcxt;
CollectedATSubcmd *newsub;
@@ -1589,6 +1593,7 @@ EventTriggerCollectAlterTableSubcmd(Node *subcmd, ObjectAddress address)
newsub->address = address;
newsub->parsetree = copyObject(subcmd);
+ currentEventTriggerState->currentCommand->d.alterTable.rewrite |= rewrite;
currentEventTriggerState->currentCommand->d.alterTable.subcmds =
lappend(currentEventTriggerState->currentCommand->d.alterTable.subcmds, newsub);
@@ -2174,3 +2179,210 @@ stringify_adefprivs_objtype(ObjectType objtype)
return "???"; /* keep compiler quiet */
}
+
+/*
+ * publication_deparse_ddl_command_start
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_start(PG_FUNCTION_ARGS)
+{
+ EventTriggerData *trigdata;
+ char *command = psprintf("Drop table command start");
+ DropStmt *stmt;
+ ListCell *cell1;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ /* extract the relid from the parse tree */
+ foreach(cell1, stmt->objects)
+ {
+ char relpersist;
+ Node *object = lfirst(cell1);
+ ObjectAddress address;
+ Relation relation = NULL;
+
+ address = get_object_address(stmt->removeType,
+ object,
+ &relation,
+ AccessExclusiveLock,
+ true);
+
+ relpersist = get_rel_persistence(address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged
+ * tables can also be created and altered on the subscriber side. This
+ * makes it possible to directly replay the SET LOGGED command and the
+ * incoming rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ {
+ table_close(relation, NoLock);
+ continue;
+ }
+
+ LogLogicalDDLMessage("deparse", address.objectId, DCT_TableDropStart,
+ command, strlen(command) + 1);
+
+ if (relation)
+ table_close(relation, NoLock);
+ }
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_table_rewrite
+ *
+ * Deparse the ddl table rewrite command and log it.
+ */
+Datum
+publication_deparse_table_rewrite(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+
+ Assert(cmd && cmd->d.alterTable.rewrite);
+
+ relpersist = get_rel_persistence(cmd->d.alterTable.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.alterTable.objectId, DCT_TableAlter,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * publication_deparse_ddl_command_end
+ *
+ * Deparse the ddl command and log it.
+ */
+Datum
+publication_deparse_ddl_command_end(PG_FUNCTION_ARGS)
+{
+ ListCell *lc;
+ slist_iter iter;
+ DeparsedCommandType type;
+ Oid relid;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ foreach(lc, currentEventTriggerState->commandList)
+ {
+ char relpersist = RELPERSISTENCE_PERMANENT;
+ CollectedCommand *cmd = lfirst(lc);
+ char *json_string;
+
+ /* Rewrite DDL has been handled in table_rewrite trigger */
+ if (cmd->d.alterTable.rewrite)
+ continue;
+
+ if (cmd->type == SCT_Simple &&
+ !OidIsValid(cmd->d.simple.address.objectId))
+ continue;
+
+ if (cmd->type == SCT_AlterTable)
+ {
+ relid = cmd->d.alterTable.objectId;
+ type = DCT_TableAlter;
+ }
+ else
+ {
+ /* Only SCT_Simple for now */
+ relid = cmd->d.simple.address.objectId;
+ type = DCT_SimpleCmd;
+ }
+
+ if (get_rel_relkind(relid))
+ relpersist = get_rel_persistence(relid);
+
+ /*
+ * Do not generate wal log for commands whose target table is a
+ * temporary table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ continue;
+
+ /*
+ * Deparse the DDL command and WAL log it to allow decoding of the
+ * same.
+ */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", relid, type, json_string,
+ strlen(json_string) + 1);
+ }
+
+ slist_foreach(iter, &(currentEventTriggerState->SQLDropList))
+ {
+ volatile SQLDropObject *obj;
+ DropStmt *stmt;
+ EventTriggerData *trigdata;
+ char *command;
+ DeparsedCommandType cmdtype;
+
+ trigdata = (EventTriggerData *) fcinfo->context;
+ stmt = (DropStmt *) trigdata->parsetree;
+
+ obj = slist_container(SQLDropObject, next, iter.cur);
+
+ if (strncmp(obj->objecttype, "table", 5) == 0)
+ cmdtype = DCT_TableDropEnd;
+ else if (strncmp(obj->objecttype, "sequence", 5) == 0 ||
+ strncmp(obj->objecttype, "schema", 5) == 0 ||
+ strncmp(obj->objecttype, "index", 5) == 0)
+ cmdtype = DCT_ObjectDrop;
+ else
+ continue;
+
+ command = deparse_drop_command(obj->objidentity, obj->objecttype,
+ stmt->behavior);
+
+ if (command == NULL)
+ continue;
+
+ LogLogicalDDLMessage("deparse", obj->address.objectId, cmdtype,
+ command, strlen(command) + 1);
+ }
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 89a0055..53bdc77 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -37,10 +37,12 @@
#include "commands/publicationcmds.h"
#include "funcapi.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "parser/parse_clause.h"
#include "parser/parse_collate.h"
#include "parser/parse_relation.h"
+#include "parser/parser.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
#include "utils/array.h"
@@ -95,6 +97,7 @@ parse_publication_options(ParseState *pstate,
pubactions->pubupdate = true;
pubactions->pubdelete = true;
pubactions->pubtruncate = true;
+ pubactions->pubddl = false;
*publish_via_partition_root = false;
/* Parse options */
@@ -141,6 +144,8 @@ parse_publication_options(ParseState *pstate,
pubactions->pubdelete = true;
else if (strcmp(publish_opt, "truncate") == 0)
pubactions->pubtruncate = true;
+ else if (strcmp(publish_opt, "ddl") == 0)
+ pubactions->pubddl = true;
else
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
@@ -749,6 +754,53 @@ CheckPubRelationColumnList(List *tables, const char *queryString,
}
/*
+ * Create event trigger which is used for DDL replication.
+ */
+static void
+CreateDDLReplicaEventTrigger(char *eventname, CommandTag *commands,
+ int ncommands, ObjectAddress pubaddress,
+ Oid puboid)
+{
+ int i;
+ List *tags = NIL;
+ Oid trigger_id;
+ ObjectAddress referenced;
+ CreateEventTrigStmt *ddl_trigger;
+ char trigger_name[NAMEDATALEN];
+ char trigger_func_name[NAMEDATALEN];
+ static const char *trigger_name_prefix = "pg_deparse_trig_%s_%u";
+ static const char *trigger_func_prefix = "publication_deparse_%s";
+
+ ddl_trigger = makeNode(CreateEventTrigStmt);
+
+ snprintf(trigger_name, sizeof(trigger_name), trigger_name_prefix,
+ eventname, puboid);
+ snprintf(trigger_func_name, sizeof(trigger_func_name), trigger_func_prefix,
+ eventname);
+
+ ddl_trigger->trigname = pstrdup(trigger_name);
+ ddl_trigger->eventname = eventname;
+ ddl_trigger->funcname = SystemFuncName(trigger_func_name);
+
+ for (i = 0; i < ncommands; i++)
+ {
+ String *tag = makeString(pstrdup(GetCommandTagName(commands[i])));
+ tags = lappend(tags, tag);
+ }
+
+ ddl_trigger->whenclause = list_make1(makeDefElem("tag", (Node *) tags, -1));
+
+ trigger_id = CreateEventTrigger(ddl_trigger);
+
+ /*
+ * Register the event triggers as internally dependent on the
+ * publication.
+ */
+ ObjectAddressSet(referenced, EventTriggerRelationId, trigger_id);
+ recordDependencyOn(&referenced, &pubaddress, DEPENDENCY_INTERNAL);
+}
+
+/*
* Create new publication.
*/
ObjectAddress
@@ -820,6 +872,8 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
BoolGetDatum(pubactions.pubdelete);
values[Anum_pg_publication_pubtruncate - 1] =
BoolGetDatum(pubactions.pubtruncate);
+ values[Anum_pg_publication_pubddl - 1] =
+ BoolGetDatum(pubactions.pubddl);
values[Anum_pg_publication_pubviaroot - 1] =
BoolGetDatum(publish_via_partition_root);
@@ -882,6 +936,61 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
}
}
+ /*
+ * Create an event trigger to allow logging of DDL statements.
+ *
+ * TODO: We need to find a better syntax to allow replication of DDL
+ * statements.
+ *
+ * XXX: This code is just to show the replication of CREATE/ALTER/DROP
+ * TABLE works. We need to enhance this once the approach for DDL
+ * replication is finalized.
+ */
+ if (pubactions.pubddl)
+ {
+ CommandTag start_commands[] = {CMDTAG_DROP_TABLE};
+ CommandTag rewrite_commands[] = {CMDTAG_ALTER_TABLE};
+
+ CommandTag end_commands[] = {
+ CMDTAG_CREATE_OPERATOR,
+ CMDTAG_CREATE_OPERATOR_CLASS,
+ CMDTAG_CREATE_OPERATOR_FAMILY,
+ CMDTAG_ALTER_OPERATOR_FAMILY,
+ CMDTAG_ALTER_OPERATOR_CLASS,
+ CMDTAG_ALTER_OPERATOR,
+ CMDTAG_CREATE_PROCEDURE,
+ CMDTAG_ALTER_PROCEDURE,
+ CMDTAG_CREATE_FUNCTION,
+ CMDTAG_ALTER_FUNCTION,
+ CMDTAG_CREATE_TRIGGER,
+ CMDTAG_ALTER_TRIGGER,
+ CMDTAG_DROP_TABLE,
+ CMDTAG_CREATE_TABLE,
+ CMDTAG_ALTER_TABLE,
+ CMDTAG_CREATE_SEQUENCE,
+ CMDTAG_ALTER_SEQUENCE,
+ CMDTAG_DROP_SEQUENCE,
+ CMDTAG_CREATE_SCHEMA,
+ CMDTAG_ALTER_SCHEMA,
+ CMDTAG_DROP_SCHEMA,
+ CMDTAG_CREATE_INDEX,
+ CMDTAG_DROP_INDEX,
+ CMDTAG_ALTER_INDEX
+ };
+
+ /* Create the ddl_command_end event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
+ lengthof(end_commands), myself, puboid);
+
+ /* Create the ddl_command_start event trigger */
+ CreateDDLReplicaEventTrigger("ddl_command_start", start_commands,
+ lengthof(start_commands), myself, puboid);
+
+ /* Create the table_rewrite event trigger */
+ CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
+ lengthof(rewrite_commands), myself, puboid);
+ }
+
table_close(rel, RowExclusiveLock);
InvokeObjectPostCreateHook(PublicationRelationId, puboid, 0);
@@ -1022,6 +1131,9 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt,
values[Anum_pg_publication_pubtruncate - 1] = BoolGetDatum(pubactions.pubtruncate);
replaces[Anum_pg_publication_pubtruncate - 1] = true;
+
+ values[Anum_pg_publication_pubddl - 1] = BoolGetDatum(pubactions.pubddl);
+ replaces[Anum_pg_publication_pubddl - 1] = true;
}
if (publish_via_partition_root_given)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d22dd44..9736de4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -5231,7 +5231,7 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab,
* Report the subcommand to interested event triggers.
*/
if (cmd)
- EventTriggerCollectAlterTableSubcmd((Node *) cmd, address);
+ EventTriggerCollectAlterTableSubcmd((Node *) cmd, address, tab->rewrite);
/*
* Bump the command counter to ensure the next subcommand in the sequence
diff --git a/src/backend/replication/logical/Makefile b/src/backend/replication/logical/Makefile
index c4e2fde..f3eeb67 100644
--- a/src/backend/replication/logical/Makefile
+++ b/src/backend/replication/logical/Makefile
@@ -16,6 +16,7 @@ override CPPFLAGS := -I$(srcdir) $(CPPFLAGS)
OBJS = \
decode.o \
+ ddlmessage.o\
launcher.o \
logical.o \
logicalfuncs.o \
diff --git a/src/backend/replication/logical/ddlmessage.c b/src/backend/replication/logical/ddlmessage.c
new file mode 100644
index 0000000..5093523
--- /dev/null
+++ b/src/backend/replication/logical/ddlmessage.c
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * ddlmessage.c
+ * Logical DDL messages.
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/replication/logical/ddlmessage.c
+ *
+ * NOTES
+ *
+ * Logical DDL messages allow XLOG logging of DDL command strings that
+ * get passed to the logical decoding plugin. In normal XLOG processing they
+ * are same as NOOP.
+ *
+ * Unlike generic logical messages, these DDL messages have only transactional
+ * mode.Note by default DDLs in PostgreSQL are transactional.
+ *
+ * These messages are part of current transaction and will be sent to
+ * decoding plugin using in a same way as DML operations.
+ *
+ * Every message carries prefix to avoid conflicts between different decoding
+ * plugins. The plugin authors must take extra care to use unique prefix,
+ * good options seems to be for example to use the name of the extension.
+ *
+ * ---------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/xact.h"
+#include "access/xloginsert.h"
+#include "catalog/namespace.h"
+#include "miscadmin.h"
+#include "nodes/execnodes.h"
+#include "replication/logical.h"
+#include "replication/ddlmessage.h"
+#include "utils/memutils.h"
+
+/*
+ * Write logical decoding DDL message into XLog.
+ */
+XLogRecPtr
+LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *message, size_t size)
+{
+ xl_logical_ddl_message xlrec;
+
+ /*
+ * Ensure we have a valid transaction id.
+ */
+ Assert(IsTransactionState());
+ GetCurrentTransactionId();
+
+ xlrec.dbId = MyDatabaseId;
+ /* trailing zero is critical; see logicalddlmsg_desc */
+ xlrec.prefix_size = strlen(prefix) + 1;
+ xlrec.message_size = size;
+ xlrec.relid = relid;
+ xlrec.cmdtype = cmdtype;
+
+ XLogBeginInsert();
+ XLogRegisterData((char *) &xlrec, SizeOfLogicalDDLMessage);
+ XLogRegisterData(unconstify(char *, prefix), xlrec.prefix_size);
+ XLogRegisterData(unconstify(char *, message), size);
+
+ /* allow origin filtering */
+ XLogSetRecordFlags(XLOG_INCLUDE_ORIGIN);
+
+ return XLogInsert(RM_LOGICALDDLMSG_ID, XLOG_LOGICAL_DDL_MESSAGE);
+}
+
+/*
+ * Redo is basically just noop for logical decoding ddl messages.
+ */
+void
+logicalddlmsg_redo(XLogReaderState *record)
+{
+ uint8 info = XLogRecGetInfo(record) & ~XLR_INFO_MASK;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(PANIC, "logicalddlmsg_redo: unknown op code %u", info);
+
+ /* This is only interesting for logical decoding, see decode.c. */
+}
diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c
index c5c6a2b..e4e0b62 100644
--- a/src/backend/replication/logical/decode.c
+++ b/src/backend/replication/logical/decode.c
@@ -36,6 +36,7 @@
#include "access/xlogutils.h"
#include "catalog/pg_control.h"
#include "replication/decode.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/message.h"
#include "replication/origin.h"
@@ -604,6 +605,46 @@ logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
}
/*
+ * Handle rmgr LOGICALDDLMSG_ID records for DecodeRecordIntoReorderBuffer().
+ */
+void
+logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
+{
+ SnapBuild *builder = ctx->snapshot_builder;
+ XLogReaderState *r = buf->record;
+ TransactionId xid = XLogRecGetXid(r);
+ uint8 info = XLogRecGetInfo(r) & ~XLR_INFO_MASK;
+ RepOriginId origin_id = XLogRecGetOrigin(r);
+ xl_logical_ddl_message *message;
+
+ if (info != XLOG_LOGICAL_DDL_MESSAGE)
+ elog(ERROR, "unexpected RM_LOGICALDDLMSG_ID record type: %u", info);
+
+ ReorderBufferProcessXid(ctx->reorder, XLogRecGetXid(r), buf->origptr);
+
+ /*
+ * If we don't have snapshot or we are just fast-forwarding, there is no
+ * point in decoding ddl messages.
+ */
+ if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT ||
+ ctx->fast_forward)
+ return;
+
+ message = (xl_logical_ddl_message *) XLogRecGetData(r);
+
+ if (message->dbId != ctx->slot->data.database ||
+ FilterByOrigin(ctx, origin_id))
+ return;
+
+ if (SnapBuildProcessChange(builder, xid, buf->origptr))
+ ReorderBufferQueueDDLMessage(ctx->reorder, xid, buf->endptr,
+ message->message, /* first part of message is prefix */
+ message->message_size,
+ message->message + message->prefix_size,
+ message->relid, message->cmdtype);
+}
+
+/*
* Consolidated commit record handling between the different form of commit
* records.
*
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.c
index 625a7f4..98969c7 100644
--- a/src/backend/replication/logical/logical.c
+++ b/src/backend/replication/logical/logical.c
@@ -33,6 +33,7 @@
#include "fmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/decode.h"
#include "replication/logical.h"
#include "replication/origin.h"
@@ -73,6 +74,10 @@ static void truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
static void message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn, const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
/* streaming callbacks */
static void stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
@@ -90,6 +95,11 @@ static void stream_change_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn
static void stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr message_lsn, bool transactional,
const char *prefix, Size message_size, const char *message);
+static void stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid, DeparsedCommandType cmdtype,
+ Size message_size, const char *message);
static void stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[], ReorderBufferChange *change);
@@ -218,6 +228,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->apply_truncate = truncate_cb_wrapper;
ctx->reorder->commit = commit_cb_wrapper;
ctx->reorder->message = message_cb_wrapper;
+ ctx->reorder->ddlmessage = ddlmessage_cb_wrapper;
/*
* To support streaming, we require start/stop/abort/commit/change
@@ -234,6 +245,7 @@ StartupDecodingContext(List *output_plugin_options,
(ctx->callbacks.stream_commit_cb != NULL) ||
(ctx->callbacks.stream_change_cb != NULL) ||
(ctx->callbacks.stream_message_cb != NULL) ||
+ (ctx->callbacks.stream_ddlmessage_cb != NULL) ||
(ctx->callbacks.stream_truncate_cb != NULL);
/*
@@ -251,6 +263,7 @@ StartupDecodingContext(List *output_plugin_options,
ctx->reorder->stream_commit = stream_commit_cb_wrapper;
ctx->reorder->stream_change = stream_change_cb_wrapper;
ctx->reorder->stream_message = stream_message_cb_wrapper;
+ ctx->reorder->stream_ddlmessage = stream_ddlmessage_cb_wrapper;
ctx->reorder->stream_truncate = stream_truncate_cb_wrapper;
@@ -1221,6 +1234,44 @@ message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ if (ctx->callbacks.ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.ddlmessage_cb(ctx, txn, message_lsn, prefix, relid, cmdtype,
+ message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_start_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
XLogRecPtr first_lsn)
{
@@ -1536,6 +1587,48 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
}
static void
+stream_ddlmessage_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message)
+{
+ LogicalDecodingContext *ctx = cache->private_data;
+ LogicalErrorCallbackState state;
+ ErrorContextCallback errcallback;
+
+ Assert(!ctx->fast_forward);
+
+ /* We're only supposed to call this when streaming is supported. */
+ Assert(ctx->streaming);
+
+ /* this callback is optional */
+ if (ctx->callbacks.stream_ddlmessage_cb == NULL)
+ return;
+
+ /* Push callback + info on the error context stack */
+ state.ctx = ctx;
+ state.callback_name = "stream_ddlmessage";
+ state.report_location = message_lsn;
+ errcallback.callback = output_plugin_error_callback;
+ errcallback.arg = (void *) &state;
+ errcallback.previous = error_context_stack;
+ error_context_stack = &errcallback;
+
+ /* set output state */
+ ctx->accept_writes = true;
+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
+ ctx->write_location = message_lsn;
+
+ /* do the actual work: call callback */
+ ctx->callbacks.stream_ddlmessage_cb(ctx, txn, message_lsn, prefix, relid,
+ cmdtype, message_size, message);
+
+ /* Pop the error context stack */
+ error_context_stack = errcallback.previous;
+}
+
+static void
stream_truncate_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
int nrelations, Relation relations[],
ReorderBufferChange *change)
diff --git a/src/backend/replication/logical/proto.c b/src/backend/replication/logical/proto.c
index ff8513e..f354062 100644
--- a/src/backend/replication/logical/proto.c
+++ b/src/backend/replication/logical/proto.c
@@ -663,6 +663,52 @@ logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
}
/*
+ * Read DDL MESSAGE from stream
+ */
+char *
+logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn,
+ const char **prefix,
+ Size *sz)
+{
+ uint8 flags;
+ char *msg;
+
+ //TODO double check when do we need to get TransactionId.
+
+ flags = pq_getmsgint(in, 1);
+ if (flags != 0)
+ elog(ERROR, "unrecognized flags %u in ddl message", flags);
+ *lsn = pq_getmsgint64(in);
+ *prefix = pq_getmsgstring(in);
+ *sz = pq_getmsgint(in, 4);
+ msg = (char *) pq_getmsgbytes(in, *sz);
+
+ return msg;
+}
+
+/*
+ * Write DDL MESSAGE to stream
+ */
+void
+logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message)
+{
+ uint8 flags = 0;
+
+ pq_sendbyte(out, LOGICAL_REP_MSG_DDLMESSAGE);
+
+ /* transaction ID (if not valid, we're not streaming) */
+ if (TransactionIdIsValid(xid))
+ pq_sendint32(out, xid);
+
+ pq_sendint8(out, flags);
+ pq_sendint64(out, lsn);
+ pq_sendstring(out, prefix);
+ pq_sendint32(out, sz);
+ pq_sendbytes(out, message, sz);
+}
+
+/*
* Write relation description to the output stream.
*/
void
@@ -1218,6 +1264,8 @@ logicalrep_message_type(LogicalRepMsgType action)
return "TYPE";
case LOGICAL_REP_MSG_MESSAGE:
return "MESSAGE";
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ return "DDL";
case LOGICAL_REP_MSG_BEGIN_PREPARE:
return "BEGIN PREPARE";
case LOGICAL_REP_MSG_PREPARE:
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 88a37fd..1f6d4ea 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -94,6 +94,7 @@
#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pgstat.h"
+#include "replication/ddlmessage.h"
#include "replication/logical.h"
#include "replication/reorderbuffer.h"
#include "replication/slot.h"
@@ -512,6 +513,14 @@ ReorderBufferReturnChange(ReorderBuffer *rb, ReorderBufferChange *change,
pfree(change->data.msg.message);
change->data.msg.message = NULL;
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ if (change->data.ddlmsg.prefix != NULL)
+ pfree(change->data.ddlmsg.prefix);
+ change->data.ddlmsg.prefix = NULL;
+ if (change->data.ddlmsg.message != NULL)
+ pfree(change->data.ddlmsg.message);
+ change->data.ddlmsg.message = NULL;
+ break;
case REORDER_BUFFER_CHANGE_INVALIDATION:
if (change->data.inval.invalidations)
pfree(change->data.inval.invalidations);
@@ -867,6 +876,36 @@ ReorderBufferQueueMessage(ReorderBuffer *rb, TransactionId xid,
}
/*
+ * A transactional DDL message is queued to be processed upon commit.
+ */
+void
+ReorderBufferQueueDDLMessage(ReorderBuffer *rb, TransactionId xid,
+ XLogRecPtr lsn, const char *prefix,
+ Size message_size, const char *message,
+ Oid relid, DeparsedCommandType cmdtype)
+{
+ MemoryContext oldcontext;
+ ReorderBufferChange *change;
+
+ Assert(xid != InvalidTransactionId);
+
+ oldcontext = MemoryContextSwitchTo(rb->context);
+
+ change = ReorderBufferGetChange(rb);
+ change->action = REORDER_BUFFER_CHANGE_DDLMESSAGE;
+ change->data.ddlmsg.prefix = pstrdup(prefix);
+ change->data.ddlmsg.relid = relid;
+ change->data.ddlmsg.cmdtype = cmdtype;
+ change->data.ddlmsg.message_size = message_size;
+ change->data.ddlmsg.message = palloc(message_size);
+ memcpy(change->data.ddlmsg.message, message, message_size);
+
+ ReorderBufferQueueChange(rb, xid, lsn, change, false);
+
+ MemoryContextSwitchTo(oldcontext);
+}
+
+/*
* AssertTXNLsnOrder
* Verify LSN ordering of transaction lists in the reorderbuffer
*
@@ -1958,6 +1997,29 @@ ReorderBufferApplyMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
}
/*
+ * Helper function for ReorderBufferProcessTXN for applying the DDL message.
+ */
+static inline void
+ReorderBufferApplyDDLMessage(ReorderBuffer *rb, ReorderBufferTXN *txn,
+ ReorderBufferChange *change, bool streaming)
+{
+ if (streaming)
+ rb->stream_ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+ else
+ rb->ddlmessage(rb, txn, change->lsn,
+ change->data.ddlmsg.prefix,
+ change->data.ddlmsg.relid,
+ change->data.ddlmsg.cmdtype,
+ change->data.ddlmsg.message_size,
+ change->data.ddlmsg.message);
+}
+
+/*
* Function to store the command id and snapshot at the end of the current
* stream so that we can reuse the same while sending the next stream.
*/
@@ -2335,6 +2397,10 @@ ReorderBufferProcessTXN(ReorderBuffer *rb, ReorderBufferTXN *txn,
ReorderBufferApplyMessage(rb, txn, change, streaming);
break;
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ ReorderBufferApplyDDLMessage(rb, txn, change, streaming);
+ break;
+
case REORDER_BUFFER_CHANGE_INVALIDATION:
/* Execute the invalidation messages locally */
ReorderBufferExecuteInvalidations(change->data.inval.ninvalidations,
@@ -3710,6 +3776,40 @@ ReorderBufferSerializeChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ char *data;
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Oid) + sizeof(int) + sizeof(Size);
+ ReorderBufferSerializeReserve(rb, sz);
+
+ data = ((char *) rb->outbuf) + sizeof(ReorderBufferDiskChange);
+
+ /* might have been reallocated above */
+ ondisk = (ReorderBufferDiskChange *) rb->outbuf;
+
+ /* write the prefix, relid and cmdtype including the size */
+ memcpy(data, &prefix_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, &change->data.ddlmsg.relid, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(data, &change->data.ddlmsg.cmdtype, sizeof(int));
+ data += sizeof(int);
+ memcpy(data, change->data.ddlmsg.prefix,
+ prefix_size);
+ data += prefix_size;
+
+ /* write the message including the size */
+ memcpy(data, &change->data.ddlmsg.message_size, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(data, change->data.ddlmsg.message,
+ change->data.ddlmsg.message_size);
+ data += change->data.ddlmsg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
char *data;
@@ -4024,6 +4124,15 @@ ReorderBufferChangeSize(ReorderBufferChange *change)
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size = strlen(change->data.ddlmsg.prefix) + 1;
+
+ sz += prefix_size + change->data.ddlmsg.message_size +
+ sizeof(Size) + sizeof(Size) + sizeof(Oid) + sizeof(int);
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
sz += sizeof(SharedInvalidationMessage) *
@@ -4299,6 +4408,33 @@ ReorderBufferRestoreChange(ReorderBuffer *rb, ReorderBufferTXN *txn,
break;
}
+ case REORDER_BUFFER_CHANGE_DDLMESSAGE:
+ {
+ Size prefix_size;
+
+ /* read prefix */
+ memcpy(&prefix_size, data, sizeof(Size));
+ data += sizeof(Size);
+ memcpy(&change->data.ddlmsg.relid, data, sizeof(Oid));
+ data += sizeof(Oid);
+ memcpy(&change->data.ddlmsg.cmdtype, data, sizeof(int));
+ data += sizeof(int);
+ change->data.ddlmsg.prefix = MemoryContextAlloc(rb->context, prefix_size);
+ memcpy(change->data.ddlmsg.prefix, data, prefix_size);
+ Assert(change->data.ddlmsg.prefix[prefix_size - 1] == '\0');
+ data += prefix_size;
+
+ /* read the message */
+ memcpy(&change->data.msg.message_size, data, sizeof(Size));
+ data += sizeof(Size);
+ change->data.msg.message = MemoryContextAlloc(rb->context,
+ change->data.msg.message_size);
+ memcpy(change->data.msg.message, data,
+ change->data.msg.message_size);
+ data += change->data.msg.message_size;
+
+ break;
+ }
case REORDER_BUFFER_CHANGE_INVALIDATION:
{
Size inval_size = sizeof(SharedInvalidationMessage) *
diff --git a/src/backend/replication/logical/worker.c b/src/backend/replication/logical/worker.c
index 5f8c541..443a3c2 100644
--- a/src/backend/replication/logical/worker.c
+++ b/src/backend/replication/logical/worker.c
@@ -156,6 +156,7 @@
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/optimizer.h"
+#include "parser/analyze.h"
#include "pgstat.h"
#include "postmaster/bgworker.h"
#include "postmaster/interrupt.h"
@@ -179,7 +180,10 @@
#include "storage/lmgr.h"
#include "storage/proc.h"
#include "storage/procarray.h"
+#include "tcop/ddl_deparse.h"
+#include "tcop/pquery.h"
#include "tcop/tcopprot.h"
+#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/catcache.h"
@@ -2463,6 +2467,230 @@ apply_handle_truncate(StringInfo s)
end_replication_step();
}
+/* Remove the data population from the command */
+static void
+preprocess_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+
+ commandTag = CreateCommandTag((Node *)command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE_AS:
+ case CMDTAG_SELECT_INTO:
+ {
+ CreateTableAsStmt *castmt = (CreateTableAsStmt *) command->stmt;
+ if (castmt->objtype == OBJECT_TABLE)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ castmt->into->skipData = true;
+ }
+ }
+ break;
+ case CMDTAG_SELECT:
+ {
+ SelectStmt *sstmt = (SelectStmt *) command->stmt;
+
+ if (sstmt->intoClause != NULL)
+ {
+ /*
+ * Force skipping data population to avoid data
+ * inconsistency. Data should be replicated from the
+ * publisher instead.
+ */
+ sstmt->intoClause->skipData = true;
+ }
+ }
+ break;
+ default:
+ break;
+ }
+}
+
+/*
+ * Handle CREATE TABLE command
+ *
+ * Call AddSubscriptionRelState for CREATE TABEL command to set the relstate to
+ * SUBREL_STATE_READY so DML changes on this new table can be replicated without
+ * having to manually run "alter subscription ... refresh publication"
+ */
+static void
+handle_create_table(RawStmt *command)
+{
+ CommandTag commandTag;
+ RangeVar *rv = NULL;
+ Oid relid;
+ Oid relnamespace = InvalidOid;
+ char *schemaname = NULL;
+ char *relname = NULL;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ switch (commandTag)
+ {
+ case CMDTAG_CREATE_TABLE:
+ {
+ CreateStmt *cstmt = (CreateStmt *) command->stmt;
+ rv = cstmt->relation;
+ }
+ break;
+ default:
+ break;
+ }
+
+ if (!rv)
+ return;
+
+ schemaname = rv->schemaname;
+ relname = rv->relname;
+
+ if (schemaname != NULL)
+ relnamespace = get_namespace_oid(schemaname, false);
+
+ if (relnamespace != InvalidOid)
+ relid = get_relname_relid(relname, relnamespace);
+ else
+ relid = RelnameGetRelid(relname);
+
+ if (relid != InvalidOid)
+ {
+ AddSubscriptionRelState(MySubscription->oid, relid,
+ SUBREL_STATE_READY,
+ InvalidXLogRecPtr);
+ ereport(DEBUG1,
+ (errmsg_internal("table \"%s\" added to subscription \"%s\"",
+ relname, MySubscription->name)));
+ }
+}
+
+static void
+apply_handle_ddl(StringInfo s)
+{
+ XLogRecPtr lsn;
+ const char *prefix = NULL;
+ char *message = NULL;
+ char *ddl_command;
+ Size sz;
+ List *parsetree_list;
+ ListCell *parsetree_item;
+ DestReceiver *receiver;
+ MemoryContext oldcontext;
+ const char *save_debug_query_string = debug_query_string;
+
+ message = logicalrep_read_ddlmessage(s, &lsn, &prefix, &sz);
+
+ /* Make sure we are in a transaction command */
+ begin_replication_step();
+
+ ddl_command = ddl_deparse_json_to_string(message);
+ debug_query_string = ddl_command;
+
+ /* DestNone for logical replication */
+ receiver = CreateDestReceiver(DestNone);
+ parsetree_list = pg_parse_query(ddl_command);
+
+ foreach(parsetree_item, parsetree_list)
+ {
+ List *plantree_list;
+ List *querytree_list;
+ RawStmt *command = (RawStmt *) lfirst(parsetree_item);
+ CommandTag commandTag;
+ MemoryContext per_parsetree_context = NULL;
+ Portal portal;
+ bool snapshot_set = false;
+
+ commandTag = CreateCommandTag((Node *) command);
+
+ /* If we got a cancel signal in parsing or prior command, quit */
+ CHECK_FOR_INTERRUPTS();
+
+ /* Remove data population from the command */
+ preprocess_create_table(command);
+
+ /*
+ * Set up a snapshot if parse analysis/planning will need one.
+ */
+ if (analyze_requires_snapshot(command))
+ {
+ PushActiveSnapshot(GetTransactionSnapshot());
+ snapshot_set = true;
+ }
+
+ /*
+ * We do the work for each parsetree in a short-lived context, to
+ * limit the memory used when there are many commands in the string.
+ */
+ per_parsetree_context =
+ AllocSetContextCreate(CurrentMemoryContext,
+ "execute_sql_string per-statement context",
+ ALLOCSET_DEFAULT_SIZES);
+ oldcontext = MemoryContextSwitchTo(per_parsetree_context);
+
+ querytree_list = pg_analyze_and_rewrite_fixedparams(command,
+ ddl_command,
+ NULL, 0, NULL);
+
+ plantree_list = pg_plan_queries(querytree_list, ddl_command, 0, NULL);
+
+ /* Done with the snapshot used for parsing/planning */
+ if (snapshot_set)
+ PopActiveSnapshot();
+
+ portal = CreatePortal("logical replication", true, true);
+
+ /*
+ * We don't have to copy anything into the portal, because everything
+ * we are passing here is in ApplyMessageContext or the
+ * per_parsetree_context, and so will outlive the portal anyway.
+ */
+ PortalDefineQuery(portal,
+ NULL,
+ ddl_command,
+ commandTag,
+ plantree_list,
+ NULL);
+
+ /*
+ * Start the portal. No parameters here.
+ */
+ PortalStart(portal, NULL, 0, InvalidSnapshot);
+
+ /*
+ * Switch back to transaction context for execution.
+ */
+ MemoryContextSwitchTo(oldcontext);
+
+ (void) PortalRun(portal,
+ FETCH_ALL,
+ true,
+ true,
+ receiver,
+ receiver,
+ NULL);
+
+ PortalDrop(portal, false);
+
+ CommandCounterIncrement();
+
+ /*
+ * Table created by DDL replication (database level) is automatically
+ * added to the subscription here.
+ */
+ handle_create_table(command);
+
+ /* Now we may drop the per-parsetree context, if one was created. */
+ MemoryContextDelete(per_parsetree_context);
+ }
+
+ debug_query_string = save_debug_query_string;
+ end_replication_step();
+}
+
/*
* Logical replication protocol message dispatcher.
@@ -2528,6 +2756,10 @@ apply_dispatch(StringInfo s)
*/
break;
+ case LOGICAL_REP_MSG_DDLMESSAGE:
+ apply_handle_ddl(s);
+ break;
+
case LOGICAL_REP_MSG_STREAM_START:
apply_handle_stream_start(s);
break;
diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c
index a3c1ba8..7bf63dc 100644
--- a/src/backend/replication/pgoutput/pgoutput.c
+++ b/src/backend/replication/pgoutput/pgoutput.c
@@ -53,6 +53,11 @@ static void pgoutput_message(LogicalDecodingContext *ctx,
ReorderBufferTXN *txn, XLogRecPtr message_lsn,
bool transactional, const char *prefix,
Size sz, const char *message);
+static void pgoutput_ddlmessage(LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn, XLogRecPtr message_lsn,
+ const char *prefix, Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz, const char *message);
static bool pgoutput_origin_filter(LogicalDecodingContext *ctx,
RepOriginId origin_id);
static void pgoutput_begin_prepare_txn(LogicalDecodingContext *ctx,
@@ -256,6 +261,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->change_cb = pgoutput_change;
cb->truncate_cb = pgoutput_truncate;
cb->message_cb = pgoutput_message;
+ cb->ddlmessage_cb = pgoutput_ddlmessage;
cb->commit_cb = pgoutput_commit_txn;
cb->begin_prepare_cb = pgoutput_begin_prepare_txn;
@@ -272,6 +278,7 @@ _PG_output_plugin_init(OutputPluginCallbacks *cb)
cb->stream_commit_cb = pgoutput_stream_commit;
cb->stream_change_cb = pgoutput_change;
cb->stream_message_cb = pgoutput_message;
+ cb->stream_ddlmessage_cb = pgoutput_ddlmessage;
cb->stream_truncate_cb = pgoutput_truncate;
/* transaction streaming - two-phase commit */
cb->stream_prepare_cb = pgoutput_stream_prepare_txn;
@@ -426,6 +433,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* This plugin uses binary protocol. */
opt->output_type = OUTPUT_PLUGIN_BINARY_OUTPUT;
+ opt->receive_rewrites = true;
/*
* This is replication start and not slot initialization.
@@ -499,6 +507,7 @@ pgoutput_startup(LogicalDecodingContext *ctx, OutputPluginOptions *opt,
/* Init publication state. */
data->publications = NIL;
+ data->deleted_relids = NIL;
publications_valid = false;
CacheRegisterSyscacheCallback(PUBLICATIONOID,
publication_invalidation_cb,
@@ -1381,9 +1390,22 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ReorderBufferChangeType action = change->action;
TupleTableSlot *old_slot = NULL;
TupleTableSlot *new_slot = NULL;
+ bool table_rewrite = false;
update_replication_progress(ctx, false);
+ /*
+ * For heap rewrites, we might need to replicate them if the rewritten
+ * table publishes rewrite ddl message. So get the actual relation here and
+ * check the pubaction later.
+ */
+ if (relation->rd_rel->relrewrite)
+ {
+ table_rewrite = true;
+ relation = RelationIdGetRelation(relation->rd_rel->relrewrite);
+ targetrel = relation;
+ }
+
if (!is_publishable_relation(relation))
return;
@@ -1417,6 +1439,13 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
Assert(false);
}
+ /*
+ * We don't publish table rewrite change unless we publish the rewrite ddl
+ * message.
+ */
+ if (table_rewrite && !relentry->pubactions.pubddl)
+ return;
+
/* Avoid leaking memory by using and resetting our own context */
old = MemoryContextSwitchTo(data->context);
@@ -1446,8 +1475,8 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
}
/* Check row filter */
- if (!pgoutput_row_filter(targetrel, NULL, &new_slot, relentry,
- &action))
+ if (!table_rewrite &&
+ !pgoutput_row_filter(targetrel, NULL, &new_slot, relentry, &action))
break;
/*
@@ -1467,8 +1496,19 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
maybe_send_schema(ctx, change, relation, relentry);
OutputPluginPrepareWrite(ctx, true);
- logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
- data->binary, relentry->columns);
+
+ /*
+ * Convert the rewrite inserts to updates so that the subscriber
+ * can replay it. This is needed to make sure the data between
+ * publisher and subscriber is consistent.
+ */
+ if (table_rewrite)
+ logicalrep_write_update(ctx->out, xid, targetrel,
+ NULL, new_slot, data->binary,
+ relentry->columns);
+ else
+ logicalrep_write_insert(ctx->out, xid, targetrel, new_slot,
+ data->binary, relentry->columns);
OutputPluginWrite(ctx, true);
break;
case REORDER_BUFFER_CHANGE_UPDATE:
@@ -1598,6 +1638,9 @@ pgoutput_change(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
ancestor = NULL;
}
+ if (table_rewrite)
+ RelationClose(relation);
+
/* Cleanup */
MemoryContextSwitchTo(old);
MemoryContextReset(data->context);
@@ -1675,8 +1718,8 @@ pgoutput_truncate(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
static void
pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
- XLogRecPtr message_lsn, bool transactional, const char *prefix, Size sz,
- const char *message)
+ XLogRecPtr message_lsn, bool transactional,
+ const char *prefix, Size sz, const char *message)
{
PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
TransactionId xid = InvalidTransactionId;
@@ -1716,6 +1759,137 @@ pgoutput_message(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
OutputPluginWrite(ctx, true);
}
+static void
+pgoutput_ddlmessage(LogicalDecodingContext *ctx, ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ Size sz, const char *message)
+{
+ PGOutputData *data = (PGOutputData *) ctx->output_plugin_private;
+ PGOutputTxnData *txndata = (PGOutputTxnData *) txn->output_plugin_private;
+ Relation relation = NULL;
+ TransactionId xid = InvalidTransactionId;
+ RelationSyncEntry *relentry;
+
+ /*
+ * Remember the xid for the message in streaming mode. See
+ * pgoutput_change.
+ */
+ if (in_streaming)
+ xid = txn->xid;
+
+ switch (cmdtype)
+ {
+ case DCT_TableDropStart:
+ /*
+ * On DROP start, add the relid to a deleted_relid list if the
+ * relid is part of a publication that supports ddl publication. We
+ * need this because on DROP end, the relid will no longer be
+ * valid. Later on Drop end, verify that the drop is for a relid
+ * that is on the deleted_rid list, and only then send the ddl
+ * message.
+ */
+ relation = RelationIdGetRelation(relid);
+
+ Assert(relation);
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (relentry->pubactions.pubddl)
+ data->deleted_relids = lappend_oid(data->deleted_relids, relid);
+
+ RelationClose(relation);
+ return;
+ case DCT_TableDropEnd:
+ if (!list_member_oid(data->deleted_relids, relid))
+ return;
+ else
+ data->deleted_relids = list_delete_oid(data->deleted_relids, relid);
+ break;
+ case DCT_TableAlter:
+ /*
+ * For table rewrite ddl, we first send the original ddl message to
+ * subscriber, then convert the upcoming rewrite INSERT to UPDATE and
+ * send them to subscriber so that the data between publisher and
+ * subscriber can always be consistent.
+ *
+ * We do this way because of two reason:
+ *
+ * (1) The data before the rewrite ddl could already be different among
+ * publisher and subscriber. To make sure the extra data in subscriber
+ * which doesn't exist in publisher also get rewritten, we need to let
+ * the subscriber execute the original rewrite ddl to rewrite all the
+ * data at first.
+ *
+ * (2) the data after executing rewrite ddl could be different among
+ * publisher and subscriber(due to different functions/operators used
+ * during rewrite), so we need to replicate the rewrite UPDATEs to keep
+ * the data consistent.
+ *
+ * TO IMPROVE: We could improve this by letting the subscriber only
+ * rewrite the extra data instead of doing fully rewrite and use the
+ * upcoming rewrite UPDATEs to rewrite the rest data. Besides, we may
+ * not need to send rewrite changes for all type of rewrite ddl, for
+ * example, it seems fine to skip sending rewrite changes for ALTER
+ * TABLE SET LOGGED as the data in the table doesn't actually be
+ * changed.
+ */
+ relation = RelationIdGetRelation(relid);
+ Assert(relation);
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ /*
+ * Skip sending this ddl if we don't publish ddl message or the ddl
+ * need to be published via its root relation.
+ */
+ if (!relentry->pubactions.pubddl ||
+ relentry->publish_as_relid != relid)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_SimpleCmd:
+ relation = RelationIdGetRelation(relid);
+
+ if (relation == NULL)
+ break;
+
+ relentry = get_rel_sync_entry(data, relation);
+
+ if (!relentry->pubactions.pubddl)
+ {
+ RelationClose(relation);
+ return;
+ }
+
+ break;
+ case DCT_ObjectDrop:
+ /* do nothing */
+ break;
+ default:
+ elog(ERROR, "unsupported type %d", cmdtype);
+ break;
+ }
+
+ /* Send BEGIN if we haven't yet */
+ if (txndata && !txndata->sent_begin_txn)
+ pgoutput_send_begin(ctx, txn);
+
+ OutputPluginPrepareWrite(ctx, true);
+ logicalrep_write_ddlmessage(ctx->out,
+ xid,
+ message_lsn,
+ prefix,
+ sz,
+ message);
+ OutputPluginWrite(ctx, true);
+
+ if (relation)
+ RelationClose(relation);
+}
+
/*
* Return true if the data is associated with an origin and the user has
* requested the changes that don't have an origin, false otherwise.
@@ -1997,7 +2171,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->schema_sent = false;
entry->streamed_txns = NIL;
entry->pubactions.pubinsert = entry->pubactions.pubupdate =
- entry->pubactions.pubdelete = entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubdelete = entry->pubactions.pubtruncate =
+ entry->pubactions.pubddl = false;
entry->new_slot = NULL;
entry->old_slot = NULL;
memset(entry->exprstate, 0, sizeof(entry->exprstate));
@@ -2055,6 +2230,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate = false;
entry->pubactions.pubdelete = false;
entry->pubactions.pubtruncate = false;
+ entry->pubactions.pubddl = false;
/*
* Tuple slots cleanups. (Will be rebuilt later if needed).
@@ -2168,6 +2344,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation)
entry->pubactions.pubupdate |= pub->pubactions.pubupdate;
entry->pubactions.pubdelete |= pub->pubactions.pubdelete;
entry->pubactions.pubtruncate |= pub->pubactions.pubtruncate;
+ entry->pubactions.pubddl |= pub->pubactions.pubddl;
/*
* We want to publish the changes as the top-most ancestor
@@ -2353,6 +2530,7 @@ rel_sync_cache_publication_cb(Datum arg, int cacheid, uint32 hashvalue)
{
entry->replicate_valid = false;
}
+
}
/* Send Replication origin */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 00dc0f2..f1b4d09 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -5683,6 +5683,7 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc)
pubdesc->pubactions.pubupdate |= pubform->pubupdate;
pubdesc->pubactions.pubdelete |= pubform->pubdelete;
pubdesc->pubactions.pubtruncate |= pubform->pubtruncate;
+ pubdesc->pubactions.pubddl |= pubform->pubddl;
/*
* Check if all columns referenced in the filter expression are part
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index da66051..e548e3c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3896,6 +3896,7 @@ getPublications(Archive *fout, int *numPublications)
int i_pubupdate;
int i_pubdelete;
int i_pubtruncate;
+ int i_pubddl;
int i_pubviaroot;
int i,
ntups;
@@ -3911,23 +3912,29 @@ getPublications(Archive *fout, int *numPublications)
resetPQExpBuffer(query);
/* Get the publications. */
- if (fout->remoteVersion >= 130000)
+ if (fout->remoteVersion >= 150000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, p.pubddl, p.pubviaroot "
+ "FROM pg_publication p");
+ else if (fout->remoteVersion >= 130000)
+ appendPQExpBuffer(query,
+ "SELECT p.tableoid, p.oid, p.pubname, "
+ "p.pubowner, "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, p.pubviaroot "
"FROM pg_publication p");
else if (fout->remoteVersion >= 110000)
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, p.pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
else
appendPQExpBuffer(query,
"SELECT p.tableoid, p.oid, p.pubname, "
"p.pubowner, "
- "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false AS pubviaroot "
+ "p.puballtables, p.pubinsert, p.pubupdate, p.pubdelete, false AS pubtruncate, false as p.pubddl, false AS pubviaroot "
"FROM pg_publication p");
res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
@@ -3943,6 +3950,7 @@ getPublications(Archive *fout, int *numPublications)
i_pubupdate = PQfnumber(res, "pubupdate");
i_pubdelete = PQfnumber(res, "pubdelete");
i_pubtruncate = PQfnumber(res, "pubtruncate");
+ i_pubddl = PQfnumber(res, "pubddl");
i_pubviaroot = PQfnumber(res, "pubviaroot");
pubinfo = pg_malloc(ntups * sizeof(PublicationInfo));
@@ -3966,6 +3974,8 @@ getPublications(Archive *fout, int *numPublications)
(strcmp(PQgetvalue(res, i, i_pubdelete), "t") == 0);
pubinfo[i].pubtruncate =
(strcmp(PQgetvalue(res, i, i_pubtruncate), "t") == 0);
+ pubinfo[i].pubddl =
+ (strcmp(PQgetvalue(res, i, i_pubddl), "t") == 0);
pubinfo[i].pubviaroot =
(strcmp(PQgetvalue(res, i, i_pubviaroot), "t") == 0);
@@ -4045,6 +4055,15 @@ dumpPublication(Archive *fout, const PublicationInfo *pubinfo)
first = false;
}
+ if (pubinfo->pubddl)
+ {
+ if (!first)
+ appendPQExpBufferStr(query, ", ");
+
+ appendPQExpBufferStr(query, "ddl");
+ first = false;
+ }
+
appendPQExpBufferStr(query, "'");
if (pubinfo->pubviaroot)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 69ee939..6c55de3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -620,6 +620,7 @@ typedef struct _PublicationInfo
bool pubdelete;
bool pubtruncate;
bool pubviaroot;
+ bool pubddl;
} PublicationInfo;
/*
diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c
index 6b8c17b..792f438 100644
--- a/src/bin/pg_waldump/rmgrdesc.c
+++ b/src/bin/pg_waldump/rmgrdesc.c
@@ -27,6 +27,7 @@
#include "commands/sequence.h"
#include "commands/tablespace.h"
#include "replication/message.h"
+#include "replication/ddlmessage.h"
#include "replication/origin.h"
#include "rmgrdesc.h"
#include "storage/standbydefs.h"
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 327a694..77c4711 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -6159,7 +6159,7 @@ listPublications(const char *pattern)
PQExpBufferData buf;
PGresult *res;
printQueryOpt myopt = pset.popt;
- static const bool translate_columns[] = {false, false, false, false, false, false, false, false};
+ static const bool translate_columns[] = {false, false, false, false, false, false, false, false, false};
if (pset.sversion < 100000)
{
@@ -6194,6 +6194,10 @@ listPublications(const char *pattern)
appendPQExpBuffer(&buf,
",\n pubviaroot AS \"%s\"",
gettext_noop("Via root"));
+ if (pset.sversion >= 140000)
+ appendPQExpBuffer(&buf,
+ ",\n pubddl AS \"%s\"",
+ gettext_noop("DDLs"));
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6284,6 +6288,7 @@ describePublications(const char *pattern)
PGresult *res;
bool has_pubtruncate;
bool has_pubviaroot;
+ bool has_pubddl;
PQExpBufferData title;
printTableContent cont;
@@ -6300,6 +6305,7 @@ describePublications(const char *pattern)
has_pubtruncate = (pset.sversion >= 110000);
has_pubviaroot = (pset.sversion >= 130000);
+ has_pubddl = (pset.sversion >= 150000);
initPQExpBuffer(&buf);
@@ -6313,6 +6319,9 @@ describePublications(const char *pattern)
if (has_pubviaroot)
appendPQExpBufferStr(&buf,
", pubviaroot");
+ if (has_pubddl)
+ appendPQExpBufferStr(&buf,
+ ", pubddl");
appendPQExpBufferStr(&buf,
"\nFROM pg_catalog.pg_publication\n");
@@ -6364,6 +6373,8 @@ describePublications(const char *pattern)
ncols++;
if (has_pubviaroot)
ncols++;
+ if (has_pubddl)
+ ncols++;
initPQExpBuffer(&title);
printfPQExpBuffer(&title, _("Publication %s"), pubname);
@@ -6378,6 +6389,8 @@ describePublications(const char *pattern)
printTableAddHeader(&cont, gettext_noop("Truncates"), true, align);
if (has_pubviaroot)
printTableAddHeader(&cont, gettext_noop("Via root"), true, align);
+ if (has_pubddl)
+ printTableAddHeader(&cont, gettext_noop("DDLs"), true, align);
printTableAddCell(&cont, PQgetvalue(res, i, 2), false, false);
printTableAddCell(&cont, PQgetvalue(res, i, 3), false, false);
@@ -6388,6 +6401,8 @@ describePublications(const char *pattern)
printTableAddCell(&cont, PQgetvalue(res, i, 7), false, false);
if (has_pubviaroot)
printTableAddCell(&cont, PQgetvalue(res, i, 8), false, false);
+ if (has_pubddl)
+ printTableAddCell(&cont, PQgetvalue(res, i, 9), false, false);
if (!puballtables)
{
diff --git a/src/include/access/rmgrlist.h b/src/include/access/rmgrlist.h
index 9a74721..9de3b8f 100644
--- a/src/include/access/rmgrlist.h
+++ b/src/include/access/rmgrlist.h
@@ -47,3 +47,4 @@ PG_RMGR(RM_COMMIT_TS_ID, "CommitTs", commit_ts_redo, commit_ts_desc, commit_ts_i
PG_RMGR(RM_REPLORIGIN_ID, "ReplicationOrigin", replorigin_redo, replorigin_desc, replorigin_identify, NULL, NULL, NULL, NULL)
PG_RMGR(RM_GENERIC_ID, "Generic", generic_redo, generic_desc, generic_identify, NULL, NULL, generic_mask, NULL)
PG_RMGR(RM_LOGICALMSG_ID, "LogicalMessage", logicalmsg_redo, logicalmsg_desc, logicalmsg_identify, NULL, NULL, NULL, logicalmsg_decode)
+PG_RMGR(RM_LOGICALDDLMSG_ID, "LogicalDDLMessage", logicalddlmsg_redo, logicalddlmsg_desc, logicalddlmsg_identify, NULL, NULL, NULL, logicalddlmsg_decode)
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3e7f364..ab290dd 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11900,4 +11900,13 @@
{ oid => '4643', descr => 'expand json format DDL to a plain DDL command',
proname => 'ddl_deparse_expand_command', prorettype => 'text',
proargtypes => 'text', prosrc => 'ddl_deparse_expand_command' },
+{ oid => '4644', descr => 'trigger for ddl command deparse',
+ proname => 'publication_deparse_ddl_command_end', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_end' },
+{ oid => '4645', descr => 'trigger for ddl command deparse start',
+ proname => 'publication_deparse_ddl_command_start', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_ddl_command_start' },
+{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
+ proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
]
diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h
index 48205ba..606c009 100644
--- a/src/include/catalog/pg_publication.h
+++ b/src/include/catalog/pg_publication.h
@@ -54,6 +54,9 @@ CATALOG(pg_publication,6104,PublicationRelationId)
/* true if partition changes are published using root schema */
bool pubviaroot;
+
+ /* true if table creations are published */
+ bool pubddl;
} FormData_pg_publication;
/* ----------------
@@ -72,6 +75,7 @@ typedef struct PublicationActions
bool pubupdate;
bool pubdelete;
bool pubtruncate;
+ bool pubddl;
} PublicationActions;
typedef struct PublicationDesc
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 10091c3..fd2ee7f 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -71,7 +71,8 @@ extern void EventTriggerCollectSimpleCommand(ObjectAddress address,
extern void EventTriggerAlterTableStart(Node *parsetree);
extern void EventTriggerAlterTableRelid(Oid objectId);
extern void EventTriggerCollectAlterTableSubcmd(Node *subcmd,
- ObjectAddress address);
+ ObjectAddress address,
+ bool rewrite);
extern void EventTriggerAlterTableEnd(void);
extern void EventTriggerCollectGrant(InternalGrant *istmt);
diff --git a/src/include/replication/ddlmessage.h b/src/include/replication/ddlmessage.h
new file mode 100644
index 0000000..a8dca86
--- /dev/null
+++ b/src/include/replication/ddlmessage.h
@@ -0,0 +1,60 @@
+/*-------------------------------------------------------------------------
+ * ddlmessage.h
+ * Exports from replication/logical/ddlmessage.c
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * src/include/replication/ddlmessage.h
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_LOGICAL_DDL_MESSAGE_H
+#define PG_LOGICAL_DDL_MESSAGE_H
+
+#include "access/xlog.h"
+#include "access/xlogdefs.h"
+#include "access/xlogreader.h"
+#include "nodes/nodes.h"
+
+
+/*
+ * Support for keeping track of deparsed commands.
+ */
+typedef enum DeparsedCommandType
+{
+ DCT_SimpleCmd,
+ DCT_TableDropStart,
+ DCT_TableDropEnd,
+ DCT_TableAlter,
+ DCT_ObjectCreate,
+ DCT_ObjectDrop
+} DeparsedCommandType;
+
+/*
+ * Generic logical decoding DDL message wal record.
+ */
+typedef struct xl_logical_ddl_message
+{
+ Oid dbId; /* database Oid emitted from */
+ Size prefix_size; /* length of prefix */
+ Oid relid; /* id of the table */
+ DeparsedCommandType cmdtype; /* type of sql command */
+ Size message_size; /* size of the message */
+
+ /*
+ * payload, including null-terminated prefix of length prefix_size
+ */
+ char message[FLEXIBLE_ARRAY_MEMBER];
+} xl_logical_ddl_message;
+
+#define SizeOfLogicalDDLMessage (offsetof(xl_logical_ddl_message, message))
+
+extern XLogRecPtr LogLogicalDDLMessage(const char *prefix, Oid relid, DeparsedCommandType cmdtype,
+ const char *ddl_message, size_t size);
+
+/* RMGR API*/
+#define XLOG_LOGICAL_DDL_MESSAGE 0x00
+void logicalddlmsg_redo(XLogReaderState *record);
+void logicalddlmsg_desc(StringInfo buf, XLogReaderState *record);
+const char *logicalddlmsg_identify(uint8 info);
+
+#endif
diff --git a/src/include/replication/decode.h b/src/include/replication/decode.h
index 741bf65..427a7b9 100644
--- a/src/include/replication/decode.h
+++ b/src/include/replication/decode.h
@@ -27,6 +27,7 @@ extern void heap2_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void standby_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void logicalmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
+extern void logicalddlmsg_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf);
extern void LogicalDecodingProcessRecord(LogicalDecodingContext *ctx,
XLogReaderState *record);
diff --git a/src/include/replication/logicalproto.h b/src/include/replication/logicalproto.h
index a771ab8..2abccb8 100644
--- a/src/include/replication/logicalproto.h
+++ b/src/include/replication/logicalproto.h
@@ -61,6 +61,7 @@ typedef enum LogicalRepMsgType
LOGICAL_REP_MSG_RELATION = 'R',
LOGICAL_REP_MSG_TYPE = 'Y',
LOGICAL_REP_MSG_MESSAGE = 'M',
+ LOGICAL_REP_MSG_DDLMESSAGE = 'L',
LOGICAL_REP_MSG_BEGIN_PREPARE = 'b',
LOGICAL_REP_MSG_PREPARE = 'P',
LOGICAL_REP_MSG_COMMIT_PREPARED = 'K',
@@ -229,7 +230,11 @@ extern void logicalrep_write_truncate(StringInfo out, TransactionId xid,
extern List *logicalrep_read_truncate(StringInfo in,
bool *cascade, bool *restart_seqs);
extern void logicalrep_write_message(StringInfo out, TransactionId xid, XLogRecPtr lsn,
- bool transactional, const char *prefix, Size sz, const char *message);
+ bool transactional, const char *prefix,
+ Size sz, const char *message);
+extern void logicalrep_write_ddlmessage(StringInfo out, TransactionId xid, XLogRecPtr lsn,
+ const char *prefix, Size sz, const char *message);
+extern char *logicalrep_read_ddlmessage(StringInfo in, XLogRecPtr *lsn, const char **prefix, Size *sz);
extern void logicalrep_write_rel(StringInfo out, TransactionId xid,
Relation rel, Bitmapset *columns);
extern LogicalRepRelation *logicalrep_read_rel(StringInfo in);
diff --git a/src/include/replication/output_plugin.h b/src/include/replication/output_plugin.h
index b7d28d7..763e43f 100644
--- a/src/include/replication/output_plugin.h
+++ b/src/include/replication/output_plugin.h
@@ -91,6 +91,18 @@ typedef void (*LogicalDecodeMessageCB) (struct LogicalDecodingContext *ctx,
const char *message);
/*
+ * Called for the logical decoding DDL messages.
+ */
+typedef void (*LogicalDecodeDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Filter changes by origin.
*/
typedef bool (*LogicalDecodeFilterByOriginCB) (struct LogicalDecodingContext *ctx,
@@ -202,6 +214,19 @@ typedef void (*LogicalDecodeStreamMessageCB) (struct LogicalDecodingContext *ctx
const char *message);
/*
+ * Callback for streaming logical decoding DDL messages from in-progress
+ * transactions.
+ */
+typedef void (*LogicalDecodeStreamDDLMessageCB) (struct LogicalDecodingContext *ctx,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size message_size,
+ const char *message);
+
+/*
* Callback for streaming truncates from in-progress transactions.
*/
typedef void (*LogicalDecodeStreamTruncateCB) (struct LogicalDecodingContext *ctx,
@@ -221,6 +246,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeTruncateCB truncate_cb;
LogicalDecodeCommitCB commit_cb;
LogicalDecodeMessageCB message_cb;
+ LogicalDecodeDDLMessageCB ddlmessage_cb;
LogicalDecodeFilterByOriginCB filter_by_origin_cb;
LogicalDecodeShutdownCB shutdown_cb;
@@ -239,6 +265,7 @@ typedef struct OutputPluginCallbacks
LogicalDecodeStreamCommitCB stream_commit_cb;
LogicalDecodeStreamChangeCB stream_change_cb;
LogicalDecodeStreamMessageCB stream_message_cb;
+ LogicalDecodeStreamDDLMessageCB stream_ddlmessage_cb;
LogicalDecodeStreamTruncateCB stream_truncate_cb;
} OutputPluginCallbacks;
diff --git a/src/include/replication/pgoutput.h b/src/include/replication/pgoutput.h
index 0202755..83e0b1e 100644
--- a/src/include/replication/pgoutput.h
+++ b/src/include/replication/pgoutput.h
@@ -25,6 +25,7 @@ typedef struct PGOutputData
uint32 protocol_version;
List *publication_names;
List *publications;
+ List *deleted_relids;
bool binary;
bool streaming;
bool messages;
diff --git a/src/include/replication/reorderbuffer.h b/src/include/replication/reorderbuffer.h
index d109d0b..737284c 100644
--- a/src/include/replication/reorderbuffer.h
+++ b/src/include/replication/reorderbuffer.h
@@ -11,6 +11,8 @@
#include "access/htup_details.h"
#include "lib/ilist.h"
+#include "nodes/nodes.h"
+#include "replication/ddlmessage.h"
#include "storage/sinval.h"
#include "utils/hsearch.h"
#include "utils/relcache.h"
@@ -56,6 +58,7 @@ typedef enum ReorderBufferChangeType
REORDER_BUFFER_CHANGE_INSERT,
REORDER_BUFFER_CHANGE_UPDATE,
REORDER_BUFFER_CHANGE_DELETE,
+ REORDER_BUFFER_CHANGE_DDLMESSAGE,
REORDER_BUFFER_CHANGE_MESSAGE,
REORDER_BUFFER_CHANGE_INVALIDATION,
REORDER_BUFFER_CHANGE_INTERNAL_SNAPSHOT,
@@ -130,6 +133,16 @@ typedef struct ReorderBufferChange
char *message;
} msg;
+ /* DDL Message. */
+ struct
+ {
+ char *prefix;
+ Size message_size;
+ char *message;
+ Oid relid;
+ DeparsedCommandType cmdtype;
+ } ddlmsg;
+
/* New snapshot, set when action == *_INTERNAL_SNAPSHOT */
Snapshot snapshot;
@@ -430,6 +443,16 @@ typedef void (*ReorderBufferMessageCB) (ReorderBuffer *rb,
const char *prefix, Size sz,
const char *message);
+/* DDL message callback signature */
+typedef void (*ReorderBufferDDLMessageCB) (ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* begin prepare callback signature */
typedef void (*ReorderBufferBeginPrepareCB) (ReorderBuffer *rb,
ReorderBufferTXN *txn);
@@ -496,6 +519,17 @@ typedef void (*ReorderBufferStreamMessageCB) (
const char *prefix, Size sz,
const char *message);
+/* stream DDL message callback signature */
+typedef void (*ReorderBufferStreamDDLMessageCB) (
+ ReorderBuffer *rb,
+ ReorderBufferTXN *txn,
+ XLogRecPtr message_lsn,
+ const char *prefix,
+ Oid relid,
+ DeparsedCommandType cmdtype,
+ Size sz,
+ const char *message);
+
/* stream truncate callback signature */
typedef void (*ReorderBufferStreamTruncateCB) (
ReorderBuffer *rb,
@@ -541,6 +575,7 @@ struct ReorderBuffer
ReorderBufferApplyTruncateCB apply_truncate;
ReorderBufferCommitCB commit;
ReorderBufferMessageCB message;
+ ReorderBufferDDLMessageCB ddlmessage;
/*
* Callbacks to be called when streaming a transaction at prepare time.
@@ -560,6 +595,7 @@ struct ReorderBuffer
ReorderBufferStreamCommitCB stream_commit;
ReorderBufferStreamChangeCB stream_change;
ReorderBufferStreamMessageCB stream_message;
+ ReorderBufferStreamDDLMessageCB stream_ddlmessage;
ReorderBufferStreamTruncateCB stream_truncate;
/*
@@ -635,6 +671,9 @@ extern void ReorderBufferQueueChange(ReorderBuffer *, TransactionId,
extern void ReorderBufferQueueMessage(ReorderBuffer *, TransactionId, Snapshot snapshot, XLogRecPtr lsn,
bool transactional, const char *prefix,
Size message_size, const char *message);
+extern void ReorderBufferQueueDDLMessage(ReorderBuffer *, TransactionId, XLogRecPtr lsn,
+ const char *prefix, Size message_size,
+ const char *message, Oid relid, DeparsedCommandType cmdtype);
extern void ReorderBufferCommit(ReorderBuffer *, TransactionId,
XLogRecPtr commit_lsn, XLogRecPtr end_lsn,
TimestampTz commit_time, RepOriginId origin_id, XLogRecPtr origin_lsn);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index a7f5700..bfc73c2 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -5970,9 +5970,9 @@ List of schemas
(0 rows)
\dRp "no.such.publication"
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
-------+-------+------------+---------+---------+---------+-----------+----------
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+------+-------+------------+---------+---------+---------+-----------+----------+------
(0 rows)
\dRs "no.such.subscription"
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 428c1f1..1df5951 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -30,20 +30,20 @@ ERROR: conflicting or redundant options
LINE 1: ...ub_xxx WITH (publish_via_partition_root = 'true', publish_vi...
^
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | f | t | f | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | f | t | f | f | f | f
(2 rows)
ALTER PUBLICATION testpub_default SET (publish = 'insert, update, delete');
\dRp
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f
- testpub_default | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpib_ins_trunct | regress_publication_user | f | t | f | f | f | f | f
+ testpub_default | regress_publication_user | f | t | t | t | f | f | f
(2 rows)
--- adding tables
@@ -87,10 +87,10 @@ RESET client_min_messages;
-- should be able to add schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable ADD ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
Tables from schemas:
@@ -99,20 +99,20 @@ Tables from schemas:
-- should be able to drop schema from 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl1"
-- should be able to set schema to 'FOR TABLE' publication
ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test;
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test"
@@ -134,10 +134,10 @@ ERROR: relation "testpub_nopk" is not part of the publication
-- should be able to set table to schema publication
ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk;
\dRp+ testpub_forschema
- Publication testpub_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
@@ -159,10 +159,10 @@ Publications:
"testpub_foralltables"
\dRp+ testpub_foralltables
- Publication testpub_foralltables
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | t | t | t | f | f | f
+ Publication testpub_foralltables
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | t | t | t | f | f | f | f
(1 row)
DROP TABLE testpub_tbl2;
@@ -174,19 +174,19 @@ CREATE PUBLICATION testpub3 FOR TABLE testpub_tbl3;
CREATE PUBLICATION testpub4 FOR TABLE ONLY testpub_tbl3;
RESET client_min_messages;
\dRp+ testpub3
- Publication testpub3
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
"public.testpub_tbl3a"
\dRp+ testpub4
- Publication testpub4
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl3"
@@ -207,10 +207,10 @@ UPDATE testpub_parted1 SET a = 1;
-- only parent is listed as being in publication, not the partition
ALTER PUBLICATION testpub_forparted ADD TABLE testpub_parted;
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_parted"
@@ -223,10 +223,10 @@ ALTER TABLE testpub_parted DETACH PARTITION testpub_parted1;
UPDATE testpub_parted1 SET a = 1;
ALTER PUBLICATION testpub_forparted SET (publish_via_partition_root = true);
\dRp+ testpub_forparted
- Publication testpub_forparted
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | t
+ Publication testpub_forparted
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | t | f
Tables:
"public.testpub_parted"
@@ -255,10 +255,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub5 FOR TABLE testpub_rf_tbl1, testpub_rf_tbl2 WHERE (c <> 'test' AND d < 5) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -271,10 +271,10 @@ Tables:
ALTER PUBLICATION testpub5 ADD TABLE testpub_rf_tbl3 WHERE (e > 1000 AND e < 2000);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl2" WHERE ((c <> 'test'::text) AND (d < 5))
@@ -290,10 +290,10 @@ Publications:
ALTER PUBLICATION testpub5 DROP TABLE testpub_rf_tbl2;
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE ((e > 1000) AND (e < 2000))
@@ -301,10 +301,10 @@ Tables:
-- remove testpub_rf_tbl1 and add testpub_rf_tbl3 again (another WHERE expression)
ALTER PUBLICATION testpub5 SET TABLE testpub_rf_tbl3 WHERE (e > 300 AND e < 500);
\dRp+ testpub5
- Publication testpub5
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub5
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl3" WHERE ((e > 300) AND (e < 500))
@@ -337,10 +337,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax1 FOR TABLE testpub_rf_tbl1, ONLY testpub_rf_tbl3 WHERE (e < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax1
- Publication testpub_syntax1
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax1
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"public.testpub_rf_tbl3" WHERE (e < 999)
@@ -350,10 +350,10 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub_syntax2 FOR TABLE testpub_rf_tbl1, testpub_rf_schema1.testpub_rf_tbl5 WHERE (h < 999) WITH (publish = 'insert');
RESET client_min_messages;
\dRp+ testpub_syntax2
- Publication testpub_syntax2
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | f | f
+ Publication testpub_syntax2
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | f | f | f
Tables:
"public.testpub_rf_tbl1"
"testpub_rf_schema1.testpub_rf_tbl5" WHERE (h < 999)
@@ -676,10 +676,10 @@ CREATE PUBLICATION testpub_table_ins WITH (publish = 'insert, truncate');
RESET client_min_messages;
ALTER PUBLICATION testpub_table_ins ADD TABLE testpub_tbl5 (a); -- ok
\dRp+ testpub_table_ins
- Publication testpub_table_ins
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | f | f | t | f
+ Publication testpub_table_ins
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | f | f | t | f | f
Tables:
"public.testpub_tbl5" (a)
@@ -821,10 +821,10 @@ CREATE TABLE testpub_tbl_both_filters (a int, b int, c int, PRIMARY KEY (a,c));
ALTER TABLE testpub_tbl_both_filters REPLICA IDENTITY USING INDEX testpub_tbl_both_filters_pkey;
ALTER PUBLICATION testpub_both_filters ADD TABLE testpub_tbl_both_filters (a,c) WHERE (c != 1);
\dRp+ testpub_both_filters
- Publication testpub_both_filters
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_both_filters
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"public.testpub_tbl_both_filters" (a, c) WHERE (c <> 1)
@@ -1029,10 +1029,10 @@ ERROR: relation "testpub_tbl1" is already member of publication "testpub_fortbl
CREATE PUBLICATION testpub_fortbl FOR TABLE testpub_tbl1;
ERROR: publication "testpub_fortbl" already exists
\dRp+ testpub_fortbl
- Publication testpub_fortbl
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortbl
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1070,10 +1070,10 @@ Publications:
"testpub_fortbl"
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
Tables:
"pub_test.testpub_nopk"
"public.testpub_tbl1"
@@ -1151,10 +1151,10 @@ REVOKE CREATE ON DATABASE regression FROM regress_publication_user2;
DROP TABLE testpub_parted;
DROP TABLE testpub_tbl1;
\dRp+ testpub_default
- Publication testpub_default
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | f | f
+ Publication testpub_default
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- fail - must be owner of publication
@@ -1164,20 +1164,20 @@ ERROR: must be owner of publication testpub_default
RESET ROLE;
ALTER PUBLICATION testpub_default RENAME TO testpub_foo;
\dRp testpub_foo
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
--------------+--------------------------+------------+---------+---------+---------+-----------+----------
- testpub_foo | regress_publication_user | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-------------+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_foo | regress_publication_user | f | t | t | t | f | f | f
(1 row)
-- rename back to keep the rest simple
ALTER PUBLICATION testpub_foo RENAME TO testpub_default;
ALTER PUBLICATION testpub_default OWNER TO regress_publication_user2;
\dRp testpub_default
- List of publications
- Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
------------------+---------------------------+------------+---------+---------+---------+-----------+----------
- testpub_default | regress_publication_user2 | f | t | t | t | f | f
+ List of publications
+ Name | Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+-----------------+---------------------------+------------+---------+---------+---------+-----------+----------+------
+ testpub_default | regress_publication_user2 | f | t | t | t | f | f | f
(1 row)
-- adding schemas and tables
@@ -1193,19 +1193,19 @@ CREATE TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA"(id int);
SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub1_forschema FOR ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
CREATE PUBLICATION testpub2_forschema FOR ALL TABLES IN SCHEMA pub_test1, pub_test2, pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1219,44 +1219,44 @@ CREATE PUBLICATION testpub6_forschema FOR ALL TABLES IN SCHEMA "CURRENT_SCHEMA",
CREATE PUBLICATION testpub_fortable FOR TABLE "CURRENT_SCHEMA"."CURRENT_SCHEMA";
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"public"
\dRp+ testpub4_forschema
- Publication testpub4_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub4_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
\dRp+ testpub5_forschema
- Publication testpub5_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub5_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub6_forschema
- Publication testpub6_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub6_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"CURRENT_SCHEMA"
"public"
\dRp+ testpub_fortable
- Publication testpub_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"CURRENT_SCHEMA.CURRENT_SCHEMA"
@@ -1290,10 +1290,10 @@ ERROR: schema "testpub_view" does not exist
-- dropping the schema should reflect the change in publication
DROP SCHEMA pub_test3;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1301,20 +1301,20 @@ Tables from schemas:
-- renaming the schema should reflect the change in publication
ALTER SCHEMA pub_test1 RENAME to pub_test1_renamed;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1_renamed"
"pub_test2"
ALTER SCHEMA pub_test1_renamed RENAME to pub_test1;
\dRp+ testpub2_forschema
- Publication testpub2_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub2_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1322,10 +1322,10 @@ Tables from schemas:
-- alter publication add schema
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1334,10 +1334,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1346,10 +1346,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema ADD ALL TABLES IN SCHEMA pub_test1;
ERROR: schema "pub_test1" is already member of publication "testpub1_forschema"
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1357,10 +1357,10 @@ Tables from schemas:
-- alter publication drop schema
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1368,10 +1368,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test2;
ERROR: tables from schema "pub_test2" are not part of the publication
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1379,29 +1379,29 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
-- drop all schemas
ALTER PUBLICATION testpub1_forschema DROP ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
-- alter publication set multiple schema
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test2;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1410,10 +1410,10 @@ Tables from schemas:
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA non_existent_schema;
ERROR: schema "non_existent_schema" does not exist
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
"pub_test2"
@@ -1422,10 +1422,10 @@ Tables from schemas:
-- removing the duplicate schemas
ALTER PUBLICATION testpub1_forschema SET ALL TABLES IN SCHEMA pub_test1, pub_test1;
\dRp+ testpub1_forschema
- Publication testpub1_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub1_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1504,18 +1504,18 @@ SET client_min_messages = 'ERROR';
CREATE PUBLICATION testpub3_forschema;
RESET client_min_messages;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
(1 row)
ALTER PUBLICATION testpub3_forschema SET ALL TABLES IN SCHEMA pub_test1;
\dRp+ testpub3_forschema
- Publication testpub3_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub3_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables from schemas:
"pub_test1"
@@ -1525,20 +1525,20 @@ CREATE PUBLICATION testpub_forschema_fortable FOR ALL TABLES IN SCHEMA pub_test1
CREATE PUBLICATION testpub_fortable_forschema FOR TABLE pub_test2.tbl1, ALL TABLES IN SCHEMA pub_test1;
RESET client_min_messages;
\dRp+ testpub_forschema_fortable
- Publication testpub_forschema_fortable
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_forschema_fortable
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
"pub_test1"
\dRp+ testpub_fortable_forschema
- Publication testpub_fortable_forschema
- Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root
---------------------------+------------+---------+---------+---------+-----------+----------
- regress_publication_user | f | t | t | t | t | f
+ Publication testpub_fortable_forschema
+ Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root | DDLs
+--------------------------+------------+---------+---------+---------+-----------+----------+------
+ regress_publication_user | f | t | t | t | t | f | f
Tables:
"pub_test2.tbl1"
Tables from schemas:
--
2.7.2.windows.1
[application/octet-stream] v15-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch (15.1K, ../../OS0PR01MB5716009FDCCC0B50BCB14A99949D9@OS0PR01MB5716.jpnprd01.prod.outlook.com/5-v15-0003-Support-CREATE-TABLE-AS-SELECT-INTO.patch)
download | inline diff:
From c7e0dd4a5b845c7079e9def53ab1fc179393cc51 Mon Sep 17 00:00:00 2001
From: "houzj.fnst" <houzj.fnst@cn.fujitsu.com>
Date: Thu, 28 Jul 2022 10:39:17 +0800
Subject: [PATCH v15 3/4] Support CREATE TABLE AS SELECT INTO
The main idea of replicating the CREATE TABLE AS is that we deprase the CREATE
TABLE AS into a simple CREATE TABLE(without subquery) command and WAL log it
after creating the table and before writing data into the table and replicate
the incoming writes later as normal INSERTs. In this apporach, we don't execute
the subquery on subscriber so that don't need to make sure all the objects
referenced in the subquery also exists in subscriber. And This approach works
for all kind of commands(e.g. CRAETE TABLE AS [SELECT][EXECUTE][VALUES])
Introducing a new type of event trigger "table_init_write". which would be fired
for CREATE TABLE AS/SELECT INTO after creating the table and
before any other modification. we deparse the command in the table_init_write event
trigger and WAL log the deparsed json string. The walsender will send the
string to subscriber. And incoming INSERTs will also be replicated.
---
src/backend/commands/createas.c | 10 ++
src/backend/commands/ddl_deparse.c | 24 +++++
src/backend/commands/event_trigger.c | 172 ++++++++++++++++++++++++++++++++-
src/backend/commands/publicationcmds.c | 9 ++
src/backend/tcop/utility.c | 2 +
src/backend/utils/cache/evtcache.c | 2 +
src/include/catalog/pg_proc.dat | 3 +
src/include/commands/event_trigger.h | 4 +
src/include/tcop/deparse_utility.h | 9 +-
src/include/utils/evtcache.h | 3 +-
10 files changed, 232 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 9abbb6b..989e894 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -34,6 +34,7 @@
#include "catalog/namespace.h"
#include "catalog/toasting.h"
#include "commands/createas.h"
+#include "commands/event_trigger.h"
#include "commands/matview.h"
#include "commands/prepare.h"
#include "commands/tablecmds.h"
@@ -143,6 +144,15 @@ create_ctas_internal(List *attrList, IntoClause *into)
StoreViewQuery(intoRelationAddr.objectId, query, false);
CommandCounterIncrement();
}
+ else
+ {
+ /*
+ * Fire the trigger for table_init_write after creating the table so
+ * that we can access the catalog info about the newly created table in
+ * the trigger function.
+ */
+ EventTriggerTableInitWrite((Node *) create, intoRelationAddr);
+ }
return intoRelationAddr;
}
diff --git a/src/backend/commands/ddl_deparse.c b/src/backend/commands/ddl_deparse.c
index bd109a6..78fb1cb 100644
--- a/src/backend/commands/ddl_deparse.c
+++ b/src/backend/commands/ddl_deparse.c
@@ -4922,6 +4922,27 @@ deparse_DefineStmt(Oid objectId, Node *parsetree, ObjectAddress secondaryObj)
}
/*
+ * Deparse CREATE TABLE AS command.
+ *
+ * deparse_CreateStmt do the actual work as we deparse the final CreateStmt for
+ * CREATE TABLE AS command.
+ */
+static ObjTree *
+deparse_CreateTableAsStmt(CollectedCommand *cmd)
+{
+ Oid objectId;
+ Node *parsetree;
+
+ Assert(cmd->type == SCT_CreateTableAs);
+
+ parsetree = cmd->d.ctas.real_create;
+ objectId = cmd->d.ctas.address.objectId;
+
+ return deparse_CreateStmt(objectId, parsetree);
+}
+
+
+/*
* Handle deparsing of simple commands.
*
* This function should cover all cases handled in ProcessUtilitySlow.
@@ -5082,6 +5103,9 @@ deparse_utility_command(CollectedCommand *cmd, bool verbose_mode)
case SCT_AlterTable:
tree = deparse_AlterTableStmt(cmd);
break;
+ case SCT_CreateTableAs:
+ tree = deparse_CreateTableAsStmt(cmd);
+ break;
case SCT_AlterOpFamily:
tree = deparse_AlterOpFamily(cmd);
break;
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index a00f061..72b83a1 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -133,7 +133,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
if (strcmp(stmt->eventname, "ddl_command_start") != 0 &&
strcmp(stmt->eventname, "ddl_command_end") != 0 &&
strcmp(stmt->eventname, "sql_drop") != 0 &&
- strcmp(stmt->eventname, "table_rewrite") != 0)
+ strcmp(stmt->eventname, "table_rewrite") != 0 &&
+ strcmp(stmt->eventname, "table_init_write") != 0)
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("unrecognized event name \"%s\"",
@@ -159,7 +160,8 @@ CreateEventTrigger(CreateEventTrigStmt *stmt)
/* Validate tag list, if any. */
if ((strcmp(stmt->eventname, "ddl_command_start") == 0 ||
strcmp(stmt->eventname, "ddl_command_end") == 0 ||
- strcmp(stmt->eventname, "sql_drop") == 0)
+ strcmp(stmt->eventname, "sql_drop") == 0 ||
+ strcmp(stmt->eventname, "table_init_write") == 0)
&& tags != NULL)
validate_ddl_tags("tag", tags);
else if (strcmp(stmt->eventname, "table_rewrite") == 0
@@ -585,7 +587,8 @@ EventTriggerCommonSetup(Node *parsetree,
dbgtag = CreateCommandTag(parsetree);
if (event == EVT_DDLCommandStart ||
event == EVT_DDLCommandEnd ||
- event == EVT_SQLDrop)
+ event == EVT_SQLDrop ||
+ event == EVT_TableInitWrite)
{
if (!command_tag_event_trigger_ok(dbgtag))
elog(ERROR, "unexpected command tag \"%s\"", GetCommandTagName(dbgtag));
@@ -868,6 +871,163 @@ EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason)
CommandCounterIncrement();
}
+
+/*
+ * EventTriggerTableInitWriteStart
+ * Prepare to receive data on an CREATE TABLE AS/SELET INTO command about
+ * to be executed.
+ */
+void
+EventTriggerTableInitWriteStart(Node *parsetree)
+{
+ MemoryContext oldcxt;
+ CollectedCommand *command;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ oldcxt = MemoryContextSwitchTo(currentEventTriggerState->cxt);
+
+ command = palloc(sizeof(CollectedCommand));
+
+ command->type = SCT_CreateTableAs;
+ command->in_extension = creating_extension;
+ command->d.ctas.address = InvalidObjectAddress;
+ command->d.ctas.real_create = NULL;
+ command->parsetree = copyObject(parsetree);
+
+ command->parent = currentEventTriggerState->currentCommand;
+ currentEventTriggerState->currentCommand = command;
+
+ MemoryContextSwitchTo(oldcxt);
+}
+
+/*
+ * EventTriggerTableInitWriteEnd
+ * Finish up saving an CREATE TABLE AS/SELECT INTO command.
+ *
+ * FIXME this API isn't considering the possibility that an xact/subxact is
+ * aborted partway through. Probably it's best to add an
+ * AtEOSubXact_EventTriggers() to fix this.
+ */
+void
+EventTriggerTableInitWriteEnd(void)
+{
+ CollectedCommand *parent;
+
+ /* ignore if event trigger context not set, or collection disabled */
+ if (!currentEventTriggerState ||
+ currentEventTriggerState->commandCollectionInhibited)
+ return;
+
+ parent = currentEventTriggerState->currentCommand->parent;
+
+ pfree(currentEventTriggerState->currentCommand);
+
+ currentEventTriggerState->currentCommand = parent;
+}
+
+/*
+ * publication_deparse_table_init_write
+ *
+ * Deparse the ddl table create command and log it.
+ */
+Datum
+publication_deparse_table_init_write(PG_FUNCTION_ARGS)
+{
+ char relpersist;
+ CollectedCommand *cmd;
+ char *json_string;
+
+ if (!CALLED_AS_EVENT_TRIGGER(fcinfo))
+ elog(ERROR, "not fired by event trigger manager");
+
+ cmd = currentEventTriggerState->currentCommand;
+ Assert(cmd);
+
+ relpersist = get_rel_persistence(cmd->d.simple.address.objectId);
+
+ /*
+ * Do not generate wal log for commands whose target table is a temporary
+ * table.
+ *
+ * We will generate wal logs for unlogged tables so that unlogged tables
+ * can also be created and altered on the subscriber side. This makes it
+ * possible to directly replay the SET LOGGED command and the incoming
+ * rewrite message without creating a new table.
+ */
+ if (relpersist == RELPERSISTENCE_TEMP)
+ return PointerGetDatum(NULL);
+
+ /* Deparse the DDL command and WAL log it to allow decoding of the same. */
+ json_string = deparse_utility_command(cmd, false);
+
+ if (json_string != NULL)
+ LogLogicalDDLMessage("deparse", cmd->d.simple.address.objectId, DCT_SimpleCmd,
+ json_string, strlen(json_string) + 1);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Fire table_init_rewrite triggers.
+ */
+void
+EventTriggerTableInitWrite(Node *real_create, ObjectAddress address)
+{
+ List *runlist;
+ EventTriggerData trigdata;
+ CollectedCommand *command;
+
+ /*
+ * See EventTriggerDDLCommandStart for a discussion about why event
+ * triggers are disabled in single user mode.
+ */
+ if (!IsUnderPostmaster)
+ return;
+
+ /*
+ * Also do nothing if our state isn't set up, which it won't be if there
+ * weren't any relevant event triggers at the start of the current DDL
+ * command. This test might therefore seem optional, but it's
+ * *necessary*, because EventTriggerCommonSetup might find triggers that
+ * didn't exist at the time the command started.
+ */
+ if (!currentEventTriggerState)
+ return;
+
+ /* Do nothing if no command was collected. */
+ if (!currentEventTriggerState->currentCommand)
+ return;
+
+ command = currentEventTriggerState->currentCommand;
+
+ runlist = EventTriggerCommonSetup(command->parsetree,
+ EVT_TableInitWrite,
+ "table_init_write",
+ &trigdata);
+ if (runlist == NIL)
+ return;
+
+ /* Set the real CreateTable statment and object address */
+ command->d.ctas.real_create = real_create;
+ command->d.ctas.address = address;
+
+ /* Run the triggers. */
+ EventTriggerInvoke(runlist, &trigdata);
+
+ /* Cleanup. */
+ list_free(runlist);
+
+ /*
+ * Make sure anything the event triggers did will be visible to the main
+ * command.
+ */
+ CommandCounterIncrement();
+}
+
/*
* Invoke each event trigger in a list of event triggers.
*/
@@ -1148,7 +1308,8 @@ trackDroppedObjectsNeeded(void)
*/
return list_length(EventCacheLookup(EVT_SQLDrop)) > 0 ||
list_length(EventCacheLookup(EVT_TableRewrite)) > 0 ||
- list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0;
+ list_length(EventCacheLookup(EVT_DDLCommandEnd)) > 0 ||
+ list_length(EventCacheLookup(EVT_TableInitWrite)) > 0;
}
/*
@@ -1867,6 +2028,7 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
case SCT_AlterOpFamily:
case SCT_CreateOpClass:
case SCT_AlterTSConfig:
+ case SCT_CreateTableAs:
{
char *identity;
char *type;
@@ -1884,6 +2046,8 @@ pg_event_trigger_ddl_commands(PG_FUNCTION_ARGS)
addr = cmd->d.createopc.address;
else if (cmd->type == SCT_AlterTSConfig)
addr = cmd->d.atscfg.address;
+ else if (cmd->type == SCT_AlterTSConfig)
+ addr = cmd->d.ctas.address;
/*
* If an object was dropped in the same command we may end
diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c
index 53bdc77..3a041b2 100644
--- a/src/backend/commands/publicationcmds.c
+++ b/src/backend/commands/publicationcmds.c
@@ -978,6 +978,11 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
CMDTAG_ALTER_INDEX
};
+ CommandTag init_commands[] = {
+ CMDTAG_CREATE_TABLE_AS,
+ CMDTAG_SELECT_INTO
+ };
+
/* Create the ddl_command_end event trigger */
CreateDDLReplicaEventTrigger("ddl_command_end", end_commands,
lengthof(end_commands), myself, puboid);
@@ -989,6 +994,10 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt)
/* Create the table_rewrite event trigger */
CreateDDLReplicaEventTrigger("table_rewrite", rewrite_commands,
lengthof(rewrite_commands), myself, puboid);
+
+ /* Create the table_init_write event trigger */
+ CreateDDLReplicaEventTrigger("table_init_write", init_commands,
+ lengthof(init_commands), myself, puboid);
}
table_close(rel, RowExclusiveLock);
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 6b0a865..490b73b 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1665,8 +1665,10 @@ ProcessUtilitySlow(ParseState *pstate,
break;
case T_CreateTableAsStmt:
+ EventTriggerTableInitWriteStart(parsetree);
address = ExecCreateTableAs(pstate, (CreateTableAsStmt *) parsetree,
params, queryEnv, qc);
+ EventTriggerTableInitWriteEnd();
break;
case T_RefreshMatViewStmt:
diff --git a/src/backend/utils/cache/evtcache.c b/src/backend/utils/cache/evtcache.c
index f7f7165..7fb8cb2 100644
--- a/src/backend/utils/cache/evtcache.c
+++ b/src/backend/utils/cache/evtcache.c
@@ -167,6 +167,8 @@ BuildEventTriggerCache(void)
event = EVT_SQLDrop;
else if (strcmp(evtevent, "table_rewrite") == 0)
event = EVT_TableRewrite;
+ else if (strcmp(evtevent, "table_init_write") == 0)
+ event = EVT_TableInitWrite;
else
continue;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index ab290dd..1367372 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -11909,4 +11909,7 @@
{ oid => '4646', descr => 'trigger for ddl command deparse table rewrite',
proname => 'publication_deparse_table_rewrite', prorettype => 'event_trigger',
proargtypes => '', prosrc => 'publication_deparse_table_rewrite' },
+{ oid => '4647', descr => 'trigger for ddl command deparse table init',
+ proname => 'publication_deparse_table_init_write', prorettype => 'event_trigger',
+ proargtypes => '', prosrc => 'publication_deparse_table_init_write' },
]
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index fd2ee7f..a9e0f71 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -55,6 +55,10 @@ extern void EventTriggerDDLCommandEnd(Node *parsetree);
extern void EventTriggerSQLDrop(Node *parsetree);
extern void EventTriggerTableRewrite(Node *parsetree, Oid tableOid, int reason);
+extern void EventTriggerTableInitWriteStart(Node *parsetree);
+extern void EventTriggerTableInitWrite(Node *parsetree, ObjectAddress address);
+extern void EventTriggerTableInitWriteEnd(void);
+
extern bool EventTriggerBeginCompleteQuery(void);
extern void EventTriggerEndCompleteQuery(void);
extern bool trackDroppedObjectsNeeded(void);
diff --git a/src/include/tcop/deparse_utility.h b/src/include/tcop/deparse_utility.h
index b53294b..3d294a0 100644
--- a/src/include/tcop/deparse_utility.h
+++ b/src/include/tcop/deparse_utility.h
@@ -29,7 +29,8 @@ typedef enum CollectedCommandType
SCT_AlterOpFamily,
SCT_AlterDefaultPrivileges,
SCT_CreateOpClass,
- SCT_AlterTSConfig
+ SCT_AlterTSConfig,
+ SCT_CreateTableAs
} CollectedCommandType;
/*
@@ -101,6 +102,12 @@ typedef struct CollectedCommand
{
ObjectType objtype;
} defprivs;
+
+ struct
+ {
+ ObjectAddress address;
+ Node *real_create;
+ } ctas;
} d;
struct CollectedCommand *parent; /* when nested */
diff --git a/src/include/utils/evtcache.h b/src/include/utils/evtcache.h
index ddb67a6..1e64831 100644
--- a/src/include/utils/evtcache.h
+++ b/src/include/utils/evtcache.h
@@ -22,7 +22,8 @@ typedef enum
EVT_DDLCommandStart,
EVT_DDLCommandEnd,
EVT_SQLDrop,
- EVT_TableRewrite
+ EVT_TableRewrite,
+ EVT_TableInitWrite
} EventTriggerEvent;
typedef struct
--
2.7.2.windows.1
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-08-05 06:03 Peter Smith <smithpb2250@gmail.com>
parent: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
0 siblings, 1 reply; 449+ messages in thread
From: Peter Smith @ 2022-08-05 06:03 UTC (permalink / raw)
To: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; +Cc: Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Zheng Li <zhengli10@gmail.com>
Hi Hou-san, here are my review comments for the patch v15-0001:
======
1. Commit Message
CREATE/ALTER/DROP TABLE (*)
At first, I thought "(*)" looks like a SQL syntax element.
SUGGESTION:
CREATE/ALTER/DROP TABLE - - Note #1, Note #2
...
Note #1 – blah blah
Note #2 – yada yada
======
2. src/backend/commands/ddl_deparse.c - General
2.1
Lots of the deparse_XXX function are in random places scattered around
in this module. Since there are so many, I think it's better to have
functions arrange alphabetically to make them easier to find. (And if
there are several functions that logically "belong" together then
those should be re-named so they will be group together
alphabetically...
Same applies to other functions – not just the deparse_XXX ones
2.2
There are lots of 'tmp' (or 'tmp2') variables in this file. Sometimes
'tmp' is appropriate (or maybe 'tmpobj' would be better) but in other
cases it seems like there should be a better name than 'tmp'. Please
search all these and replace where you can use a more meaningful name
than tmp.
2.3
Pointer NULL comparisons are not done consistently all through the
file. E.g. Sometimes you do tree->fmtinfo == NULL, but in others you
do like if (!tree->fmtinfo). It's no big deal whichever way you want
to use, but at least for the comparisons involving the same variables
IMO should use the same style consistently.
~~~
3. src/backend/commands/ddl_deparse.c - format_type_detailed
3.1
+ * - typename is set to the type name, without quotes
But the param is called 'typname', not 'typename'
3.2
+ * - typmod is set to the typemod, if any, as a string with parens
I think you mean:
"typmod is set" -> "typemodstr is set"
3.3
+ if (IsTrueArrayType(typeform) &&
+ typeform->typstorage != TYPSTORAGE_PLAIN)
+ {
+ /* Switch our attention to the array element type */
+ ReleaseSysCache(tuple);
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(array_base_type));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", type_oid);
+
+ typeform = (Form_pg_type) GETSTRUCT(tuple);
+ type_oid = array_base_type;
+ *typarray = true;
+ }
+ else
+ *typarray = false;
Maybe this if/else can be simplified
*typarray = IsTrueArrayType(typeform) && typeform->typstorage !=
TYPSTORAGE_PLAIN;
if (*typarray)
{
...
}
3.4
+ /* otherwise, WITH TZ is added by typmod. */
Uppercase comment
~~~
4. src/backend/commands/ddl_deparse.c - append_object_to_format_string
+ for (cp = sub_fmt; cp < end_ptr; cp++)
+ {
+ if (*cp == '{')
+ {
+ start_copy = true;
+ continue;
+ }
What's this logic going to do if it encounters "{{" - it looks like it
will just keep going but wouldn't that be a name error to have a "{"
in it?
~~~
5. src/backend/commands/ddl_deparse.c - append_bool_object
+append_bool_object(ObjTree *tree, char *sub_fmt, bool value)
+{
+ ObjElem *param;
+ char *object_name = sub_fmt;
+ bool is_present_flag = false;
+
+ Assert(sub_fmt);
+
+ if (strcmp(sub_fmt, "present") == 0)
+ {
+ is_present_flag = true;
+ tree->present = value;
+ }
+
+ if (!verbose && !tree->present)
+ return;
+
+ if (!is_present_flag)
+ object_name = append_object_to_format_string(tree, sub_fmt);
+
+ param = new_object(ObjTypeBool, object_name);
+ param->value.boolean = value;
+ append_premade_object(tree, param);
+}
It feels like there is some subtle trickery going on here with the
conditions. Is there a simpler way to write this, or maybe it is just
lacking some explanatory comments?
~~~
6. src/backend/commands/ddl_deparse.c - append_array_object
+ if (!verbose)
+ {
+ ListCell *lc;
+
+ /* Extract the ObjElems whose present flag is true */
+ foreach(lc, array)
+ {
+ ObjElem *elem = (ObjElem *) lfirst(lc);
+
+ Assert(elem->objtype == ObjTypeObject);
+
+ if (!elem->value.object->present)
+ array = foreach_delete_current(array, lc);
+ }
+
+ if (list_length(array) == 0)
+ return;
+ }
Maybe it is OK as-is. I'm just wondering if this list_length(array)
check should be outside of the !verbose check?
~~~
7. src/backend/commands/ddl_deparse.c - objtree_to_jsonb_element
+ case ObjTypeObject:
+ /* recursively add the object into the existing parse state */
Uppercase comment
~~~
8. src/backend/commands/ddl_deparse.c - new_objtree_for_qualname_id
8.1
+ *
+ * Elements "schemaname" and "objname" are set. If the object is a temporary
+ * object, the schema name is set to "pg_temp".
I'm not sure if this is the right place to say this, since it is not
really this function that sets that "pg_temp".
8.2
+ if (isnull)
+ elog(ERROR, "unexpected NULL namespace");
+ objname = heap_getattr(catobj, Anum_name, RelationGetDescr(catalog),
+ &isnull);
Missing blank line after the elog?
~~~
9. src/backend/commands/ddl_deparse.c - deparse_ColumnIdentity
+ /* Definition elemets */
typo "elemets"
~~~
10. src/backend/commands/ddl_deparse.c - deparse_CreateTrigStmt
10.1
+ else
+ elog(ERROR, "unrecognized trigger timing value %d", node->timing);
should that say "unrecognized trigger timing type" (e.g. type instead of value)
10.2
+ /*
+ * Decode the events that the trigger fires for. The output is a list;
+ * in most cases it will just be a string with the even name, but when
+ * there's an UPDATE with a list of columns, we return a JSON object.
+ */
"even name" -> "event name" ?
10.3
+ foreach(cell, node->columns)
+ {
+ char *colname = strVal(lfirst(cell));
+
+ cols = lappend(cols,
+ new_string_object(colname));
+ }
Unnecessary wrapping?
10.4
+ append_array_object(update, "%{columns:, }I", cols);
+
+ events = lappend(events,
+ new_object_object(update));
Unnecessary wrapping?
10.5
+ /* verify that the argument encoding is correct */
Uppercase comment
~~~
11. src/backend/commands/ddl_deparse.c - deparse_ColumnDef
+ saw_notnull = false;
+ foreach(cell, coldef->constraints)
+ {
+ Constraint *constr = (Constraint *) lfirst(cell);
+
+ if (constr->contype == CONSTR_NOTNULL)
+ saw_notnull = true;
+ }
Some similar functions here would 'break' when it finds the
saw_notnull. Why not break here?
~~~
12. src/backend/commands/ddl_deparse.c - obtainConstraints
12.1
+ /* only one may be valid */
Uppercase comment
12.2
+ /*
+ * scan pg_constraint to fetch all constraints linked to the given
+ * relation.
+ */
Uppercase comment
~~~
13. src/backend/commands/ddl_deparse.c - deparse_InhRelations
+/*
+ * Deparse the inherits relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
I am not sure - should that say "Deparse the INHERITS relations." (uppercase)
~~~
14. src/backend/commands/ddl_deparse.c - pg_get_indexdef_detailed
14.1
+ * There is a huge lot of code that's a dupe of pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
SUGGESTION
A large amount of code is duplicated from pg_get_indexdef_worker, ...
14.2
+ idxrelrec = (Form_pg_class) GETSTRUCT(ht_idxrel);
+
+
+ /*
+ * Fetch the pg_am tuple of the index' access method
+ */
Remove the extra blank line
~~~
15. src/backend/commands/ddl_deparse.c - deparse_IndexStmt
+ /*
+ * indexes for PRIMARY KEY and other constraints are output
+ * separately; return empty here.
+ */
Uppercase comment
~~~
16. src/backend/commands/ddl_deparse.c - deparse_FunctionSet
16.1
+static ObjTree *
+deparse_FunctionSet(VariableSetKind kind, char *name, char *value)
Missing function comment.
16.2
+ ObjTree *r;
Is 'r' the best name?
16.3
+ if (kind == VAR_RESET_ALL)
+ {
+ r = new_objtree("RESET ALL");
+ }
+ else if (value != NULL)
+ {
+ r = new_objtree_VA("SET %{set_name}I", 1,
+ "set_name", ObjTypeString, name);
+
+ /*
+ * Some GUC variable names are 'LIST' type and hence must not be
+ * quoted.
+ */
+ if (GetConfigOptionFlags(name, true) & GUC_LIST_QUOTE)
+ append_string_object(r, "TO %{set_value}s", value);
+ else
+ append_string_object(r, "TO %{set_value}L", value);
+ }
+ else
+ {
+ r = new_objtree("RESET");
+ append_string_object(r, "%{set_name}I", name);
+ }
It seems a bit strange that the kind of new_objtree is judged
sometimes by the *value. Why isn't this just always switching on the
different VariableSetKind?
~~~
17. src/backend/commands/ddl_deparse.c - deparse_CreateFunction
17.1
+/*
+ * deparse_CreateFunctionStmt
+ * Deparse a CreateFunctionStmt (CREATE FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
+static ObjTree *
+deparse_CreateFunction(Oid objectId, Node *parsetree)
The name of the function and the name of the function in the comment
don't match. Suggest removing it from the comment.
17.2
+ /* get the pg_proc tuple */
Uppercase comment
17.3
+ /* get the corresponding pg_language tuple */
Uppercase comment
17.4
+ /* optional wholesale suppression of "name" occurs here */
Uppercase comment
17.5
+ append_string_object(createFunc, "%{volatility}s",
+ procForm->provolatile == PROVOLATILE_VOLATILE ?
+ "VOLATILE" :
+ procForm->provolatile == PROVOLATILE_STABLE ?
+ "STABLE" :
+ procForm->provolatile == PROVOLATILE_IMMUTABLE ?
+ "IMMUTABLE" : "INVALID VOLATILITY");
Does "INVALID VOLATILITY" make sense? Is that a real thing or should
this give ERROR?
17.6
+ foreach(cell, node->options)
+ {
+ DefElem *defel = (DefElem *) lfirst(cell);
+ ObjTree *tmp = NULL;
Why assign *tmp = NULL? I think it serves no purpose.
~~~
18. src/backend/commands/ddl_deparse.c - deparse_AlterFunction
18.1
+ * Deparse a AlterFunctionStmt (ALTER FUNCTION)
"a" -> "an"
18.2
+ /* get the pg_proc tuple */
Uppercase comment
18.3
+ alterFunc = new_objtree_VA("ALTER FUNCTION", 0);
+
+ params = NIL;
Why not just assign params = NIL at the declaration?
~~~
19. src/backend/commands/ddl_deparse.c - deparse_AlterOwnerStmt
+ * Deparse a AlterOwnerStmt (ALTER ... OWNER TO ...).
"a" -> "an"
~~~
20. src/backend/commands/ddl_deparse.c - deparse_AlterOperatorStmt
+ * Deparse a AlterOperatorStmt (ALTER OPERATOR ... SET ...).
"a" -> "an"
~~~
21. src/backend/commands/ddl_deparse.c - deparse_RenameStmt
21.1
+ * In a ALTER .. RENAME command, we don't have the original name of the
"a" -> "an"
21.2
+ relation_close(relation, AccessShareLock);
+
+ break;
+ case OBJECT_SCHEMA:
Misplaced bank line; should be before after break;
21.3
+ break;
+ case OBJECT_TRIGGER:
Put blank line after break;
21.4
+ break;
+ default:
Put blank line after break;
~~~
22. src/backend/commands/ddl_deparse.c - deparse_Seq_Cache
+static inline ObjElem *
+deparse_Seq_Cache(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~
23. src/backend/commands/ddl_deparse.c - deparse_Seq_Cycle
+static inline ObjElem *
+deparse_Seq_Cycle(ObjTree *parent, Form_pg_sequence seqdata, bool alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~~
24. src/backend/commands/ddl_deparse.c - deparse_Seq_IncrementBy
+static inline ObjElem *
+deparse_Seq_IncrementBy(ObjTree *parent, Form_pg_sequence seqdata,
bool alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~~
25. src/backend/commands/ddl_deparse.c - deparse_Seq_Minvalue
+static inline ObjElem *
+deparse_Seq_Minvalue(ObjTree *parent, Form_pg_sequence seqdata, bool
alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~~
26. src/backend/commands/ddl_deparse.c - deparse_Seq_Maxvalue
+static inline ObjElem *
+deparse_Seq_Maxvalue(ObjTree *parent, Form_pg_sequence seqdata, bool
alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~~
27. src/backend/commands/ddl_deparse.c - deparse_Seq_Startwith
+static inline ObjElem *
+deparse_Seq_Startwith(ObjTree *parent, Form_pg_sequence seqdata, bool
alter_table)
Is param 'alter_table’ correct here or should that be 'alter_sequence'
(or just 'alter')?
~~~
28. src/backend/commands/ddl_deparse.c - deparse_CreateSchemaStmt
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * Note we don't output the schema elements given in the creation command.
+ * They must be output separately. (In the current implementation,
+ * CreateSchemaCommand passes them back to ProcessUtility, which will lead to
+ * this file if appropriate.)
+ */
"this file" ??
~~~
29. src/backend/commands/ddl_deparse.c - deparse_Seq_OwnedBy
29.1
+static ObjElem *
+deparse_Seq_OwnedBy(ObjTree *parent, Oid sequenceId, bool alter_table)
Missing function comment.
29.2
+ /* only consider AUTO dependencies on pg_class */
Uppercase comment.
29.3
+ if (!ownedby)
+ /* XXX this shouldn't happen ... */
+ ownedby = new_objtree_VA("OWNED BY %{owner}D",
+ 3,
+ "clause", ObjTypeString, "owned",
+ "owner", ObjTypeNull,
+ "present", ObjTypeBool, false);
+ return new_object_object(ownedby);
Put blank line before return;
~~~
30. src/backend/commands/ddl_deparse.c - deparse_CreateSeqStmt
30.1
+ /* definition elemets */
Uppercase comment, and typo "elemets"
30.2
+ /* we purposefully do not emit OWNED BY here */
Uppercase comment
~~~
31. src/backend/commands/ddl_deparse.c - deparse_AlterTableStmt
31.1
+ tmp = new_objtree_VA("ADD CONSTRAINT %{name}I",
+ 2,
+ "type", ObjTypeString, "add constraint using index",
+ "name", ObjTypeString, get_constraint_name(constrOid));
I think it was customary for you to put the number of varags on the
1st line, not like this. There are several others like this in this
function which should also be changed (where they fit OK on the first
line).
31.2
+ append_array_object(alterTableStmt, "%{subcmds:, }s", subcmds);
+ return alterTableStmt;
Maybe add blank line before the return;
~~~
32. src/backend/commands/ddl_deparse.c - deparse_AlterOpFamily
32.1
+ list = NIL;
+ foreach(cell, cmd->d.opfam.operators)
Why not assign list = NIL with the declaration?
32.2
+ proargtypes = procForm->proargtypes.values;
+ arglist = NIL;
+ for (i = 0; i < procForm->pronargs; i++)
Why not assign arglist = NIL with the declaration?
32.3
+ if (!stmt->isDrop)
+ append_format_string(alterOpFam, "ADD");
+ else
+ append_format_string(alterOpFam, "DROP");
Maybe simpler to reverse these; IMO isDrop means "DROP" makes more sense.
~~~
33. src/backend/commands/ddl_deparse.c - deparse_DefineStmt_Operator
+ /* Add the definition clause */
+ list = NIL;
Why not assign list = NIL with the declaration?
~~~
34. src/backend/commands/ddl_deparse.c - deparse_DefineStmt
+ default:
+ elog(ERROR, "unsupported object kind");
+ return NULL;
What purpose does this return serve after the ERROR? If you have to do
something to quieten the compiler, maybe it's better to set defStmt =
NULL at declaration.
======
35. src/backend/commands/ddl_json.c - <General>
In the errmsg text some of the messages say JSON (uppercase) and some
say json (lowercase). IMO they should all be consistent. Maybe say
JSON, since that way seems dominant.
~~~
36. src/backend/commands/ddl_json.c - enum
+typedef enum
+{
+ tv_absent,
+ tv_true,
+ tv_false
+} trivalue;
It seems there is another enum elsewhere already called this, because
you did not add it into the typedefs.list, yet it is already there. Is
that OK? Maybe this should have a unique name for this module.
~~~
37. src/backend/commands/ddl_json.c - expand_fmt_recursive
37.1
+ is_array = false;
+
+ ADVANCE_PARSE_POINTER(cp, end_ptr);
Why not assign is_array = false in the declaration?
37.2
+ /*
+ * Validate that we got an array if the format string specified one.
+ * And finally print out the data
+ */
+ if (is_array)
+ expand_jsonb_array(buf, param, value, arraysep, specifier, start_ptr);
+ else
+ expand_one_jsonb_element(buf, param, value, specifier, start_ptr);
"Print out" the data? This comment seems a bit over-complicated.
Perhaps these sentences can be combined and re-worded a bit.
SUGGESTION (maybe?)
Expand the data (possibly an array) into the output StringInfo.
~~~
38. src/backend/commands/ddl_json.c - expand_jsonval_identifier
+/*
+ * Expand a json value as an identifier. The value must be of type string.
+ */
+static void
+expand_jsonval_identifier(StringInfo buf, JsonbValue *jsonval)
Should that say "as a quoted identifier" ?
~~~
39. src/backend/commands/ddl_json.c - expand_jsonval_typename
39.1
+ switch (is_array)
+ {
+ default:
+ case tv_absent:
It seems slightly unusual for the default case to not be the last
switch case. Consider rearranging it.
39.2
+ if (schema == NULL)
+ appendStringInfo(buf, "%s%s%s",
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
+
+ /* Special typmod needs */
+ else if (schema[0] == '\0')
+ appendStringInfo(buf, "%s%s%s",
+ typename,
+ typmodstr ? typmodstr : "",
+ array_decor);
+ else
+ appendStringInfo(buf, "%s.%s%s%s",
+ quote_identifier(schema),
+ quote_identifier(typename),
+ typmodstr ? typmodstr : "",
+ array_decor);
The last 2 parts:
typmodstr ? typmodstr : "",
array_decor);
are common for all those above appendStringInfo, so you could reduce
the code (if you want to) and just add the common parts at the end.
e.g.
if (schema == NULL)
appendStringInfo(buf, "%s", quote_identifier(typename));
else if (schema[0] == '\0')
appendStringInfo(buf, "%s", typename); /* Special typmod needs */
else
appendStringInfo(buf, "%s.%s", quote_identifier(schema),
quote_identifier(typename));
appendStringInfo(buf, "%s%s", typmodstr ? typmodstr : "", array_decor);
39.3
In other code (e.g. expand_jsonval_dottedname) you did lots of
pfree(str) after using the strings, so why not similar here?
~~~
40. src/backend/commands/ddl_json.c - expand_jsonval_operator
40.1
+ /* schema might be NULL or empty */
Uppercase comment
40.2
Why no pfree(str) here similar to what there was in prior code (e.g.
expand_jsonval_dottedname)?
~~~
41. src/backend/commands/ddl_json.c - expand_jsonval_string
Comment says "The value must be of type string or of type object."
Yeah, but what it is isn't? This code will just fall thru and return
true. Is that the right behaviour? Should there be an Assert at least?
~~~
42. src/backend/commands/ddl_json.c - expand_jsonval_number
Does this need some pfree after the string is copied to 'buf'?
~~~
43 src/backend/commands/ddl_json.c - expand_jsonval_role
+ rolename = find_string_in_jsonbcontainer(jsonval->val.binary.data,
+ "rolename", false, NULL);
+ appendStringInfoString(buf, quote_identifier(rolename));
Does this need some pfree after the string is copied to 'buf'?
~~~
44. src/backend/commands/ddl_json.c - ddl_deparse_json_to_string
+ d = DirectFunctionCall1(jsonb_in,
+ PointerGetDatum(json_str));
Seems unnecessary wrapping here.
~~~
45. src/backend/commands/ddl_json.c - fmtstr_error_callback
45.1
+/*
+ * Error context callback for JSON format string expansion.
+ *
+ * Possible improvement: indicate which element we're expanding, if applicable.
+ */
Should that "Possible improvement" comment have "XXX" prefix like most
other possible improvement comments have?
45.2
+fmtstr_error_callback(void *arg)
+{
+ errcontext("while expanding format string \"%s\"", (char *) arg);
+
+}
Remove the blank line.
======
46. src/backend/utils/adt/ruleutils.c - pg_get_trigger_whenclause
+char *
+pg_get_trigger_whenclause(Form_pg_trigger trigrec, Node *whenClause,
bool pretty)
Missing function comment
~~~
47. src/backend/utils/adt/ruleutils.c - print_function_sqlbody
@@ -3513,7 +3526,7 @@ pg_get_function_arg_default(PG_FUNCTION_ARGS)
PG_RETURN_TEXT_P(string_to_text(str));
}
-static void
+void
print_function_sqlbody(StringInfo buf, HeapTuple proctup)
{
Having a function comment is more important now that this is no longer static.
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-08-08 20:22 Zheng Li <zhengli10@gmail.com>
parent: Amit Kapila <amit.kapila16@gmail.com>
0 siblings, 1 reply; 449+ messages in thread
From: Zheng Li @ 2022-08-08 20:22 UTC (permalink / raw)
To: Amit Kapila <amit.kapila16@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
> In general, I agree with your comment below that we can work on this
> after we have some more concrete plans/discussions. I think we can
> probably consider this when we have more discussion around the
> publication commands for the DDL objects. However, it would be good if
> you can add some more details about the above two options.
Thanks for the support. I have started a new thread on supporting replication of
global object command and have added more detailed discussion:
https://www.postgresql.org/message-id/CAAD30UKD7YPEbYcs_L9PYLcLZjnxyqO%3DJF5_mnAwx7g_PtOi3A%40mail.g...
> I noticed that LogLogicalDDLMessage() uses MyDatabaseId and then
> decoding can take some action (filtering) based on that. Is it Okay to
> use that function for global objects, if so, you might want to add a
> comment for the same?
Could you elaborate on the concern? The dbid filtering happens in
logicalddlmsg_decode but I don't see why I can't use LogLogicalDDLMessage
to log global commands. Are there any global commands that are
non-transactional?
logicalddlmsg_decode:
if (message->dbId != ctx->slot->data.database ||
FilterByOrigin(ctx, origin_id))
return;
> As per my understanding, the overall work on this project includes the
> following sub-tasks:
> a. DDL Deparsing: This is required to support DDL replication of
> non-global objects. The work for this is in progress, this is based on
> prior work by Alvaro.
> b. DDL Replication: This includes replication of DDL commands based on
> event triggers and DDL deparsing. The work on this is also in
> progress.
> c. DDL Replication of global objects: It requires a different approach
> due to the reasons quoted above in your email. Zheng has started
> working on it.
> d. Initial Sync: I think there is a brief discussion about this in the
> thread but no concrete proposal yet. I think it is important to solve
> this part of the puzzle as well to have an overall design ready for
> this project. Do let me know if you, Sawada-San, or anybody else
> intends to work on it? I think that will avoid overlap of work.
Euler mentioned that he has plan to work on the initial schema sync in
[1]. We can help with this effort as well.
Regards,
Zheng
[1] https://www.postgresql.org/message-id/45d0d97c-3322-4054-b94f-3c08774bbd90%40www.fastmail.com
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-08-12 10:32 Amit Kapila <amit.kapila16@gmail.com>
parent: Zheng Li <zhengli10@gmail.com>
0 siblings, 0 replies; 449+ messages in thread
From: Amit Kapila @ 2022-08-12 10:32 UTC (permalink / raw)
To: Zheng Li <zhengli10@gmail.com>; +Cc: houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>; Peter Smith <smithpb2250@gmail.com>
On Tue, Aug 9, 2022 at 1:52 AM Zheng Li <zhengli10@gmail.com> wrote:
>
> > In general, I agree with your comment below that we can work on this
> > after we have some more concrete plans/discussions. I think we can
> > probably consider this when we have more discussion around the
> > publication commands for the DDL objects. However, it would be good if
> > you can add some more details about the above two options.
>
> Thanks for the support. I have started a new thread on supporting replication of
> global object command and have added more detailed discussion:
> https://www.postgresql.org/message-id/CAAD30UKD7YPEbYcs_L9PYLcLZjnxyqO%3DJF5_mnAwx7g_PtOi3A%40mail.g...
>
> > I noticed that LogLogicalDDLMessage() uses MyDatabaseId and then
> > decoding can take some action (filtering) based on that. Is it Okay to
> > use that function for global objects, if so, you might want to add a
> > comment for the same?
>
> Could you elaborate on the concern? The dbid filtering happens in
> logicalddlmsg_decode but I don't see why I can't use LogLogicalDDLMessage
> to log global commands.
>
I thought one may not want to tie replication of global objects with
any particular database. I read your ideas on the thread [1] about the
same but am not sure of the best way forward for it.
[1] - https://www.postgresql.org/message-id/CAAD30UKD7YPEbYcs_L9PYLcLZjnxyqO%3DJF5_mnAwx7g_PtOi3A%40mail.g...
--
With Regards,
Amit Kapila.
^ permalink raw reply [nested|flat] 449+ messages in thread
* Re: Support logical replication of DDLs
@ 2022-10-06 08:30 Peter Smith <smithpb2250@gmail.com>
parent: vignesh C <vignesh21@gmail.com>
1 sibling, 1 reply; 449+ messages in thread
From: Peter Smith @ 2022-10-06 08:30 UTC (permalink / raw)
To: vignesh C <vignesh21@gmail.com>; +Cc: Zheng Li <zhengli10@gmail.com>; Ajin Cherian <itsajin@gmail.com>; houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>; Amit Kapila <amit.kapila16@gmail.com>; Masahiko Sawada <sawada.mshk@gmail.com>; Japin Li <japinli@hotmail.com>; Alvaro Herrera <alvherre@alvh.no-ip.org>; Dilip Kumar <dilipbalaut@gmail.com>; rajesh singarapu <rajesh.rs0541@gmail.com>; PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
The patches here are quite large, so for this review post, I have only
done a quick check for cosmetic stuff in the comments of patch
v24-0001.
~
I did this mostly just by cutting/pasting the whole patch text into a
grammar/spell-checker to see what it reported. Please use the same
strategy prior to posting future patch versions, because it will be
way more efficient/easier for the author to spend a few minutes to fix
everything like this up-front before posting, rather than getting a
flood of review comments to deal with (like this post) about such
stuff.
(BTW, most of these suggestions are just verbatim what my
grammar/spell-checker told me)
======
1. Commit comment
(Note #2) Note that, for ATTACH/DETACH PARTITION, we haven't added
extra logic on
subscriber to handle the case where the table on publisher is a PARTITIONED
TABLE while the target table on subcriber side is NORMAL table. We will
research this more and improve this later.
SUGGESTION (minor changes + fix typo)
(Note #2) Note that for ATTACH/DETACH PARTITION we haven't added extra logic on
the subscriber to handle the case where the table on the publisher is
a PARTITIONED
TABLE while the target table on the subscriber is a NORMAL table. We will
research this more and improve it later.
======
2. GENERAL - uppercase the comments
Please make all your single-line comments start with uppercase for consistency.
Here are some examples to fix:
Line 303: + /* add the "ON table" clause */
Line 331: + /* add the USING clause, if any */
Line 349: + /* add the WITH CHECK clause, if any */
Line 653: + /* otherwise, WITH TZ is added by typmod. */
Line 663: + /* otherwise, WITH TZ is added by typmode. */
Line 1946: + /* build list of privileges to grant/revoke */
Line 2017: + /* target objects. We use object identities here */
Line 2041: + /* list of grantees */
Line 2053: + /* the wording of the grant option is variable ... */
Line 2632: + /* skip this one; we add one unconditionally below */
Line 2660: + /* done */
Line 2768: + /* add HANDLER clause */
Line 2780: + /* add VALIDATOR clause */
Line 2792: + /* add an OPTIONS clause, if any */
Line 2845: + /* add HANDLER clause */
Line 2859: + /* add VALIDATOR clause */
Line 2877: + /* add an OPTIONS clause, if any */
Line 5024: + /* add the rest of the stuff */
Line 5040: + /* add the rest of the stuff */
Line 5185: + /* a new constraint has a name and definition */
Line 5211: + /* done */
Line 6124: + /* add a CONCURRENTLY clause */
Line 6127: + /* add the matview name */
Line 6131: + /* add a WITH NO DATA clause */
Line 6302: + /* reloptions */
Line 6310: + /* tablespace */
Line 6592: + /* deconstruct the name list */
Line 6636: + /* deconstruct the name list */
Line 6725: + /* obtain object tuple */
Line 6729: + /* obtain namespace */
------
3. Grammar/Spelling
3a. - format_type_detailed
+ * - nspid is the schema OID. For certain SQL-standard types which have weird
+ * typmod rules, we return InvalidOid; caller is expected to not schema-
+ * qualify the name nor add quotes to the type name in this case.
"caller" -> "the caller"
~
3b. - format_type_detailed
+ * - typemodstr is set to the typemod, if any, as a string with parens
"parens" -> "parentheses"
~
3c. - format_type_detailed
+ else
+ /* otherwise, WITH TZ is added by typmode. */
+ *typname = pstrdup("TIME");
"typmode" -> "typmod" ?
~
3d. - new_objtree_for_qualname
+ * A helper routine to setup %{}D and %{}O elements.
"setup" -> "set up"
~
3e. - new_objtree_for_type
+/*
+ * A helper routine to setup %{}T elements.
+ */
+static ObjTree *
+new_objtree_for_type(Oid typeId, int32 typmod)
"setup" -> "set up"
~
3f. - pg_get_indexdef_detailed
+/*
+ * Return an index definition, split in several pieces.
+ *
+ * A large amount of code is duplicated from pg_get_indexdef_worker, but
+ * control flow is different enough that it doesn't seem worth keeping them
+ * together.
+ */
+static void
+pg_get_indexdef_detailed(Oid indexrelid,
"split in" -> "split into"
~
3g. - ddl_deparse_to_json
+ * The command is expanded fully, so that there are no ambiguities even in the
+ * face of search_path changes.
+ */
+Datum
+ddl_deparse_to_json(PG_FUNCTION_ARGS)
"fully, so" -> "fully so"
~
3h. -deparse_AlterFunction
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the alter command.
+ */
"the parsetree" -> "the parse tree"
~
3i. - deparse_AlterObjectSchemaStmt
+/*
+ * deparse an ALTER ... SET SCHEMA command.
+ */
+static ObjTree *
+deparse_AlterObjectSchemaStmt(ObjectAddress address, Node *parsetree,
"deparse" -> "Deparse"
~
3j. deparse_AlterObjectSchemaStmt
+ /*
+ * Since the command has already taken place from the point of view of
+ * catalogs, getObjectIdentity returns the object name with the already
+ * changed schema. The output of our deparsing must return the original
+ * schema name however, so we chop the schema name off the identity string
+ * and then prepend the quoted schema name.
"name however," -> "name, however,"
~
3k. - deparse_GrantStmt
+ /* build list of privileges to grant/revoke */
+ if (istmt->all_privs)
"build list" -> "build a list"
~
3l. - deparse_AlterTypeSetStmt
+ * Deparse an AlterTypeStmt.
+ *
+ * Given a type OID and a parsetree that modified it, return an ObjTree
+ * representing the alter type.
+ */
+static ObjTree *
+deparse_AlterTypeSetStmt(Oid objectId, Node *cmd)
"parsetree" -> "parse tree"
~
3m. - deparse_CompositeTypeStmt
+ * Deparse a CompositeTypeStmt (CREATE TYPE AS)
+ *
+ * Given a type OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3n. - deparse_CreateEnumStmt
+/*
+ * Deparse a CreateEnumStmt (CREATE TYPE AS ENUM)
+ *
+ * Given a type OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3o. - deparse_CreateExtensionStmt
+/*
+ * deparse_CreateExtensionStmt
+ * deparse a CreateExtensionStmt
+ *
+ * Given an extension OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ *
+ */
"parsetree" -> "parse tree"
~
3p. - deparse_CreateFdwStmt
+/*
+ * deparse_CreateFdwStmt
+ * Deparse a CreateFdwStmt (CREATE FOREIGN DATA WRAPPER)
+ *
+ * Given a trigger OID and the parsetree that created it,
+ * return an ObjTree representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3q. - deparse_AlterFdwStmt
+/*
+ * deparse_AlterFdwStmt
+ * Deparse an AlterFdwStmt (ALTER FOREIGN DATA WRAPPER)
+ *
+ * Given a function OID and the parsetree that create it, return the
+ * JSON blob representing the alter command.
+ */
"parsetree" -> "parse tree"
"create it" -> "created it"
3r.
+ /*
+ * Iterate through options, to see what changed, but use catalog as basis
+ * for new values.
+ */
"as basis" -> "as a basis"
~
3s.
+/*
+ * Deparse a CREATE TYPE AS RANGE statement
+ *
+ * Given a type OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3t. - deparse_AlterEnumStmt
+/*
+ * Deparse an AlterEnumStmt.
+ *
+ * Given a type OID and a parsetree that modified it, return an ObjTree
+ * representing the alter type.
+ */
"parsetree" -> "parse tree"
~
3u. - deparse_AlterTableStmt
+ /*
+ * We don't support replicating ALTER TABLE which contains volatile
+ * functions because It's possible the functions contain DDL/DML in
+ * which case these opertions will be executed twice and cause
+ * duplicate data. In addition, we don't know whether the tables being
+ * accessed by these DDL/DML are published or not. So blindly allowing
+ * such functions can allow unintended clauses like the tables accessed
+ * in those functions may not even exist on the subscriber-side.
+ */
"opertions" -> "operations"
"subscriber-side." -> "subscriber."
~
3v. - deparse_ColumnDef
+ * Deparse a ColumnDef node within a regular (non typed) table creation.
"non typed" -> "non-typed"
~
3w. - deparse_ColumnDef
+ /*
+ * Emit a NOT NULL declaration if necessary. Note that we cannot trust
+ * pg_attribute.attnotnull here, because that bit is also set when
+ * primary keys are specified; and we must not emit a NOT NULL
+ * constraint in that case, unless explicitely specified. Therefore,
+ * we scan the list of constraints attached to this column to determine
+ * whether we need to emit anything.
+ * (Fortunately, NOT NULL constraints cannot be table constraints.)
+ *
+ * In the ALTER TABLE cases, we also add a NOT NULL if the colDef is
+ * marked is_not_null.
+ */
"specified; and we" -> "specified; we"
"explicitely" -> "explicitly"
~
3x. - deparse_CreateDomain
+/*
+ * Deparse the CREATE DOMAIN
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3y. - deparse_CreateFunction
+/*
+ * Deparse a CreateFunctionStmt (CREATE FUNCTION)
+ *
+ * Given a function OID and the parsetree that created it, return the JSON
+ * blob representing the creation command.
+ */
"parsetree" -> "parse tree"
~
3z. - deparse_CreateFunction
+ /*
+ * Now iterate over each parameter in the parsetree to create the
+ * parameters array.
+ */
"parsetree" -> "parse tree"
~
4a. - deparse_CreateFunction
+ /*
+ * A PARAM_TABLE parameter indicates end of input arguments; the
+ * following parameters are part of the return type. We ignore them
+ * here, but keep track of the current position in the list so that
+ * we can easily produce the return type below.
+ */
"end" -> "the end"
~
4b. - deparse_CreateOpClassStmt
+ /* Don't deparse sql commands generated while creating extension */
+ if (cmd->in_extension)
+ return NULL;
"sql" -> "SQL"
~
4c. - deparse_CreateOpClassStmt
/*
+ * Add the FAMILY clause; but if it has the same name and namespace as the
+ * opclass, then have it expand to empty, because it would cause a failure
+ * if the opfamily was created internally.
+ */
"clause; " -> "clause, "
"empty," -> "empty"
~
4d. - deparse_CreateOpFamily
+/*
+ * Deparse a CreateTrigStmt (CREATE OPERATOR FAMILY)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
4e. - deparse_CreateSchemaStmt
+/*
+ * Deparse a CreateSchemaStmt.
+ *
+ * Given a schema OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ */
"parsetree" -> "parse tree"
~
4f. - deparse_CreateSeqStmt
+/*
+ * Deparse a CreateSeqStmt.
+ *
+ * Given a sequence OID and the parsetree that create it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
"create it" -> "created it"
~
4g. - deparse_CreateStmt
+/*
+ * Deparse a CreateStmt (CREATE TABLE).
+ *
+ * Given a table OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
4h.
+ /*
+ * Typed tables and partitions use a slightly different format string: we
+ * must not put table_elements with parents directly in the fmt string,
+ * because if there are no options the parens must not be emitted; and
+ * also, typed tables do not allow for inheritance.
+ */
"parens" -> "parentheses"
~
4i. - deparse_CreateStmt
+ /*
+ * We can't put table elements directly in the fmt string as an array
+ * surrounded by parens here, because an empty clause would cause a
+ * syntax error. Therefore, we use an indirection element and set
+ * present=false when there are no elements.
+ */
"parens" -> "parentheses"
~
4j. - deparse_CreateStmt
+ /*
+ * Get pg_class.relpartbound. We cannot use partbound in the
+ * parsetree directly as it's the original partbound expression
+ * which haven't been transformed.
+ */
"parsetree" -> "parse tree" ? maybe this one if ok if it referring to
the parameter with this name.
~
4k. - deparse_DefineStmt_Operator
+/*
+ * Deparse a DefineStmt (CREATE OPERATOR)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parse tree"
~
4l. - deparse_CreateTrigStmt
+/*
+ * Deparse a CreateTrigStmt (CREATE TRIGGER)
+ *
+ * Given a trigger OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ */
"parsetree" -> "parsetree"
~
4m. - deparse_RefreshMatViewStmt
+/*
+ * Deparse a RefreshMatViewStmt (REFRESH MATERIALIZED VIEW)
+ *
+ * Given a materialized view OID and the parsetree that created it, return an
+ * ObjTree representing the refresh command.
+ */
"parseree" -> "parse tree"
~
4n. - deparse_IndexStmt
+/*
+ * Deparse an IndexStmt.
+ *
+ * Given an index OID and the parsetree that created it, return an ObjTree
+ * representing the creation command.
+ *
+ * If the index corresponds to a constraint, NULL is returned.
+ */
"parsetree" -> "parse tree"
~
4o. - deparse_InhRelations
+/*
+ * Deparse the INHERITS relations.
+ *
+ * Given a table OID, return a schema qualified table list representing
+ * the parent tables.
+ */
+static List *
+deparse_InhRelations(Oid objectId)
"schema qualified" -> "schema-qualified"
~
4p. - deparse_OnCommitClause
+/*
+ * Deparse the ON COMMMIT ... clause for CREATE ... TEMPORARY ...
+ */
+static ObjTree *
+deparse_OnCommitClause(OnCommitAction option)
"COMMMIT" -> "COMMIT"
~
4q. - deparse_RuleStmt
+/*
+ * Deparse a RuleStmt (CREATE RULE).
+ *
+ * Given a rule OID and the parsetree that created it, return an ObjTree
+ * representing the rule command.
+ */
"parsetree" -> "parse tree"
~
4r. - deparse_utility_command
+ /*
+ * Allocate everything done by the deparsing routines into a temp context,
+ * to avoid having to sprinkle them with memory handling code; but allocate
+ * the output StringInfo before switching.
+ */
"code; " -> "code, "
~
4s. - deparse_utility_command
+ /*
+ * Many routines underlying this one will invoke ruleutils.c functionality
+ * in order to obtain deparsed versions of expressions. In such results,
+ * we want all object names to be qualified, so that results are "portable"
+ * to environments with different search_path settings. Rather than inject
+ * what would be repetitive calls to override search path all over the
+ * place, we do it centrally here.
+ */
"in order to" => "to"
~
4t. - convSpecifier
+/*
+ * Conversion specifier, it determines how we expand the json element into
+ * string.
+ */
SUGGESTION
Conversion specifier which determines how we expand the json element
into a string.
~
4u. - json_trivalue
+/*
+ * A ternary value which represents a boolean type JsonbValue.
+ */
"which represents" -> "that represents"
~
4v. - expand_fmt_recursive
+ /*
+ * Scan the mandatory element name. Allow for an array separator
+ * (which may be the empty string) to be specified after colon.
+ */
"after colon" -> "after a colon"
~
4w. - expand_jsonval_string
+/*
+ * Expand a json value as a string. The value must be of type string or of
+ * type object. In the latter case it must contain a "fmt" element which will
+ * be recursively expanded; also, if the object contains an element "present"
+ * and it is set to false, the expansion is the empty string.
+ *
+ * Returns false if no actual expansion was made due to the "present" flag
+ * being set to "false".
+ */
"latter case" -> "latter case,"
~
4x. - format_procedure_args_internal
+/*
+ * Append the parenthised arguments of the given pg_proc row into the output
+ * buffer. force_qualify indicates whether to schema-qualify type names
+ * regardless of visibility.
+ */
"parenthised" -> "parenthesized "
~
4y.
+/*
+ * Internal version that returns definition of a CONSTRAINT command
+ */
"definition" -> "the definition"
======
5. Function comment inconsistencies
5a.
Sometimes the function name is repeated in the comment and sometimes it is not.
e.g. compare deparse_CreateEnumStmt() versus deparse_CreateExtensionStmt(), etc.
(IMO there is no need to repeat the function name)
~
5b.
Sometimes the deparse function comments are verbose and say like:
+ * Given a type OID and a parsetree that modified it, return an ObjTree
+ * representing the alter type.
but sometimes - like deparse_AlterExtensionStmt() etc. - they don't
bother to say anything at all.
e.g.
+/*
+ * Deparse ALTER EXTENSION .. UPDATE TO VERSION
+ */
+static ObjTree *
+deparse_AlterExtensionStmt(Oid objectId, Node *parsetree)
Either it is useful, so say it always, or it is not useful, so say it
never. Pick one.
------
6. GENERAL - json
IMO change "json" -> "JSON" everywhere.
Here are some examples:
Line 7605: + * Conversion specifier, it determines how we expand the
json element into
Line 7699: + errmsg("missing element \"%s\" in json object", keyname)));
Line 7857: + * Expand a json value as a quoted identifier. The value
must be of type string.
Line 7872: + * Expand a json value as a dot-separated-name. The value
must be of type
Line 7908: + * Expand a json value as a type name.
Line 7966: + * Expand a json value as an operator name
Line 7993: + * Expand a json value as a string. The value must be of
type string or of
Line 8031: + * Expand a json value as a string literal.
Line 8070: + * Expand a json value as an integer quantity.
Line 8083: + * Expand a json value as a role name. If the is_public
element is set to
Line 8111: + * Expand one json element into the output StringInfo
according to the
Line 8807: +{ oid => '4642', descr => 'deparse the DDL command into
json format string',
Line 8810: +{ oid => '4643', descr => 'expand json format DDL to a
plain DDL command',
------
Kind Regards,
Peter Smith.
Fujitsu Australia
^ permalink raw reply [nested|flat] 449+ messages in thread
318 further messages in this thread omitted from this page (render size limit).
Use the per-message pages, or the t.mbox.gz byte stream, for the complete thread.
end of thread, other threads:[~2026-02-24 01:21 UTC | newest]
Thread overview: 449+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2022-02-21 15:53 Support logical replication of DDLs Zheng Li <zhengli10@gmail.com>
2022-02-22 10:05 ` Aleksander Alekseev <aleksander@timescale.com>
2022-02-24 03:10 ` Zheng Li <zhengli10@gmail.com>
2022-02-24 08:56 ` Aleksander Alekseev <aleksander@timescale.com>
2022-02-22 12:00 ` Marcos Pegoraro <marcos@f10.com.br>
2022-03-13 11:35 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-03-15 06:27 ` rajesh singarapu <rajesh.rs0541@gmail.com>
2022-03-16 21:17 ` Zheng Li <zhengli10@gmail.com>
2022-03-16 22:06 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-03-17 16:22 ` Zheng Li <zhengli10@gmail.com>
2022-03-18 00:18 ` Zheng Li <zhengli10@gmail.com>
2022-03-18 00:25 ` Japin Li <japinli@hotmail.com>
2022-03-18 10:19 ` Japin Li <japinli@hotmail.com>
2022-03-18 17:25 ` Zheng Li <zhengli10@gmail.com>
2022-03-18 17:29 ` Zheng Li <zhelli@amazon.com>
2022-03-18 23:33 ` Japin Li <japinli@hotmail.com>
2022-03-21 20:56 ` Zheng Li <zhengli10@gmail.com>
2022-03-23 05:09 ` Japin Li <japinli@hotmail.com>
2022-04-06 11:37 ` Ajin Cherian <itsajin@gmail.com>
2022-04-07 10:16 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-11 05:00 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-11 12:46 ` Euler Taveira <euler@eulerto.com>
2022-04-11 17:36 ` Zheng Li <zhengli10@gmail.com>
2022-04-12 10:55 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-13 09:07 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-04-13 09:50 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-13 17:22 ` Zheng Li <zhengli10@gmail.com>
2022-05-10 07:02 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-05-10 09:27 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-11 17:31 ` Zheng Li <zhengli10@gmail.com>
2022-04-12 11:12 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-03 21:30 ` Zheng Li <zhengli10@gmail.com>
2022-05-13 21:01 ` Zheng Li <zhengli10@gmail.com>
2022-05-26 08:35 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-05-26 22:19 ` Zheng Li <zhengli10@gmail.com>
2022-05-27 02:03 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-30 06:52 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-06-02 12:14 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-03 11:15 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-07 02:59 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-09 11:43 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-10 06:26 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-12 06:46 ` Zheng Li <zhengli10@gmail.com>
2022-06-13 10:08 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-15 00:14 ` Zheng Li <zhengli10@gmail.com>
2022-06-15 04:00 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-17 19:38 ` Zheng Li <zhengli10@gmail.com>
2022-06-20 03:31 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-21 12:19 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-22 09:07 ` vignesh C <vignesh21@gmail.com>
2022-06-26 17:50 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-06-28 03:26 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-29 09:47 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-30 06:14 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-30 08:16 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-28 12:13 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-29 03:06 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-29 09:54 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-29 10:48 ` Amit Kapila <amit.kapila16@gmail.com>
2022-07-05 08:28 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-07-08 02:26 ` Peter Smith <smithpb2250@gmail.com>
2022-07-12 09:03 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-07-20 12:28 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-07-22 21:18 ` Zheng Li <zhengli10@gmail.com>
2022-07-23 15:33 ` Joe Conway <mail@joeconway.com>
2022-07-23 16:23 ` Zheng Li <zhengli10@gmail.com>
2022-07-25 06:13 ` Amit Kapila <amit.kapila16@gmail.com>
2022-08-08 20:22 ` Zheng Li <zhengli10@gmail.com>
2022-08-12 10:32 ` Amit Kapila <amit.kapila16@gmail.com>
2022-07-28 03:48 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-08-01 04:18 ` Peter Smith <smithpb2250@gmail.com>
2022-08-02 03:20 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-08-05 06:03 ` Peter Smith <smithpb2250@gmail.com>
2022-08-13 15:26 ` Ajin Cherian <itsajin@gmail.com>
2022-08-16 11:57 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-08-29 06:14 ` Ajin Cherian <itsajin@gmail.com>
2022-09-05 14:18 ` Ajin Cherian <itsajin@gmail.com>
2022-09-15 12:38 ` Ajin Cherian <itsajin@gmail.com>
2022-09-20 14:57 ` Ajin Cherian <itsajin@gmail.com>
2022-09-21 16:55 ` Zheng Li <zhengli10@gmail.com>
2022-09-26 10:26 ` vignesh C <vignesh21@gmail.com>
2022-09-30 07:47 ` vignesh C <vignesh21@gmail.com>
2022-10-06 08:30 ` Peter Smith <smithpb2250@gmail.com>
2022-10-13 12:46 ` Ajin Cherian <itsajin@gmail.com>
2022-10-17 06:11 ` vignesh C <vignesh21@gmail.com>
2022-10-18 01:08 ` Zheng Li <zhengli10@gmail.com>
2022-10-06 16:40 ` Zheng Li <zhengli10@gmail.com>
2022-10-06 17:16 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-10-06 21:30 ` Zheng Li <zhengli10@gmail.com>
2022-10-07 00:01 ` Ajin Cherian <itsajin@gmail.com>
2022-10-07 02:40 ` Zheng Li <zhengli10@gmail.com>
2022-10-07 05:07 ` vignesh C <vignesh21@gmail.com>
2022-10-10 02:38 ` Peter Smith <smithpb2250@gmail.com>
2022-10-10 09:58 ` Ajin Cherian <itsajin@gmail.com>
2022-10-11 13:30 ` Ajin Cherian <itsajin@gmail.com>
2022-10-12 06:07 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-10-19 05:48 ` vignesh C <vignesh21@gmail.com>
2022-10-24 12:29 ` Ajin Cherian <itsajin@gmail.com>
2022-10-26 20:39 ` Zheng Li <zhengli10@gmail.com>
2022-10-27 10:32 ` vignesh C <vignesh21@gmail.com>
2022-10-31 10:47 ` vignesh C <vignesh21@gmail.com>
2022-11-01 23:43 ` vignesh C <vignesh21@gmail.com>
2022-11-04 09:36 ` vignesh C <vignesh21@gmail.com>
2022-11-11 14:39 ` vignesh C <vignesh21@gmail.com>
2022-11-14 06:33 ` vignesh C <vignesh21@gmail.com>
2022-11-08 04:03 ` Ajin Cherian <itsajin@gmail.com>
2022-11-10 04:04 ` Ajin Cherian <itsajin@gmail.com>
2022-12-07 11:22 ` vignesh C <vignesh21@gmail.com>
2022-12-07 12:20 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-12-08 05:07 ` vignesh C <vignesh21@gmail.com>
2022-12-09 03:27 ` li jie <ggysxcq@gmail.com>
2022-12-09 10:36 ` Amit Kapila <amit.kapila16@gmail.com>
2023-01-02 08:06 ` vignesh C <vignesh21@gmail.com>
2022-10-27 23:18 ` Peter Smith <smithpb2250@gmail.com>
2022-10-28 00:20 ` Zheng Li <zhengli10@gmail.com>
2022-10-28 00:38 ` Peter Smith <smithpb2250@gmail.com>
2022-10-29 09:30 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-10-28 06:50 ` Peter Smith <smithpb2250@gmail.com>
2022-10-31 08:07 ` Peter Smith <smithpb2250@gmail.com>
2022-11-03 07:40 ` Peter Smith <smithpb2250@gmail.com>
2022-11-11 04:47 ` Peter Smith <smithpb2250@gmail.com>
2022-11-11 05:09 ` Peter Smith <smithpb2250@gmail.com>
2022-11-11 05:17 ` Peter Smith <smithpb2250@gmail.com>
2022-11-11 05:33 ` Peter Smith <smithpb2250@gmail.com>
2022-11-20 03:59 ` vignesh C <vignesh21@gmail.com>
2022-11-25 06:17 ` vignesh C <vignesh21@gmail.com>
2022-11-25 09:06 ` li jie <ggysxcq@gmail.com>
2022-11-25 22:23 ` Zheng Li <zhengli10@gmail.com>
2022-11-28 03:27 ` li jie <ggysxcq@gmail.com>
2022-11-28 03:28 ` Zheng Li <zhengli10@gmail.com>
2022-11-28 04:47 ` Zheng Li <zhengli10@gmail.com>
2022-11-29 02:29 ` rajesh singarapu <rajesh.rs0541@gmail.com>
2022-11-29 04:25 ` Ajin Cherian <itsajin@gmail.com>
2022-11-29 04:39 ` rajesh singarapu <rajesh.rs0541@gmail.com>
2022-11-29 04:44 ` Ajin Cherian <itsajin@gmail.com>
2022-11-29 05:05 ` rajesh singarapu <rajesh.rs0541@gmail.com>
2022-11-29 05:22 ` rajesh singarapu <rajesh.rs0541@gmail.com>
2022-11-29 06:39 ` Ajin Cherian <itsajin@gmail.com>
2022-11-29 12:21 ` li jie <ggysxcq@gmail.com>
2022-11-29 14:35 ` vignesh C <vignesh21@gmail.com>
2022-11-29 18:31 ` vignesh C <vignesh21@gmail.com>
2022-12-01 13:03 ` li jie <ggysxcq@gmail.com>
2022-12-02 02:46 ` Zheng Li <zhengli10@gmail.com>
2022-12-09 03:31 ` li jie <ggysxcq@gmail.com>
2022-12-12 04:58 ` Zheng Li <zhengli10@gmail.com>
2022-12-06 18:52 ` vignesh C <vignesh21@gmail.com>
2022-12-02 23:48 ` Zheng Li <zhengli10@gmail.com>
2022-11-25 11:00 ` vignesh C <vignesh21@gmail.com>
2022-11-16 08:57 ` vignesh C <vignesh21@gmail.com>
2022-11-25 10:30 ` vignesh C <vignesh21@gmail.com>
2022-12-15 16:24 ` vignesh C <vignesh21@gmail.com>
2022-12-19 10:02 ` li jie <ggysxcq@gmail.com>
2022-12-19 21:25 ` Zheng Li <zhengli10@gmail.com>
2022-12-21 17:21 ` vignesh C <vignesh21@gmail.com>
2022-12-23 06:03 ` Takamichi Osumi (Fujitsu) <osumi.takamichi@fujitsu.com>
2022-12-27 06:13 ` vignesh C <vignesh21@gmail.com>
2022-12-27 12:03 ` vignesh C <vignesh21@gmail.com>
2022-12-28 22:42 ` Zheng Li <zhengli10@gmail.com>
2022-12-28 22:50 ` Zheng Li <zhengli10@gmail.com>
2023-01-04 21:35 ` Zheng Li <zhengli10@gmail.com>
2023-01-07 15:28 ` Zheng Li <zhengli10@gmail.com>
2023-01-13 06:33 ` vignesh C <vignesh21@gmail.com>
2023-01-15 04:09 ` Ajin Cherian <itsajin@gmail.com>
2023-01-16 03:12 ` vignesh C <vignesh21@gmail.com>
2023-01-16 13:33 ` Ajin Cherian <itsajin@gmail.com>
2023-01-19 08:51 ` vignesh C <vignesh21@gmail.com>
2023-02-03 00:41 ` Peter Smith <smithpb2250@gmail.com>
2023-02-03 10:21 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-03 13:27 ` Ajin Cherian <itsajin@gmail.com>
2023-02-06 07:07 ` Ajin Cherian <itsajin@gmail.com>
2023-02-05 20:49 ` Peter Smith <smithpb2250@gmail.com>
2023-02-13 09:07 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-09 09:55 ` Ajin Cherian <itsajin@gmail.com>
2023-02-09 10:55 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-16 07:54 ` Ajin Cherian <itsajin@gmail.com>
2023-02-10 11:06 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-14 11:02 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-19 14:54 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-15 03:01 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-16 16:48 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-16 17:53 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-16 19:01 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-16 19:38 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-16 19:43 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-16 21:08 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-17 11:04 ` vignesh C <vignesh21@gmail.com>
2023-02-17 09:15 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-19 02:20 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-20 04:14 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-20 23:44 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-02-10 14:52 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-02-15 05:57 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-19 14:55 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-15 04:32 ` Peter Smith <smithpb2250@gmail.com>
2023-02-15 08:32 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-15 09:51 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-15 22:16 ` Zheng Li <zhengli10@gmail.com>
2023-02-16 04:00 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-17 04:13 ` Zheng Li <zhengli10@gmail.com>
2023-02-20 08:22 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-02-21 02:09 ` Zheng Li <zhengli10@gmail.com>
2023-02-22 23:01 ` Zheng Li <zhengli10@gmail.com>
2023-02-24 08:42 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-02-16 13:15 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-16 18:02 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-03-06 06:34 ` Ajin Cherian <itsajin@gmail.com>
2023-03-06 10:16 ` wangw.fnst@fujitsu.com <wangw.fnst@fujitsu.com>
2023-03-06 20:37 ` Zheng Li <zhengli10@gmail.com>
2023-03-09 02:26 ` wangw.fnst@fujitsu.com <wangw.fnst@fujitsu.com>
2023-03-09 09:58 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-09 09:58 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-06 13:13 ` vignesh C <vignesh21@gmail.com>
2023-03-10 10:28 ` vignesh C <vignesh21@gmail.com>
2023-03-10 22:41 ` Runqi Tian <runqidev@gmail.com>
2023-03-12 15:24 ` Zheng Li <zhengli10@gmail.com>
2023-03-14 04:17 ` Ajin Cherian <itsajin@gmail.com>
2023-03-16 02:43 ` wangw.fnst@fujitsu.com <wangw.fnst@fujitsu.com>
2023-03-20 09:17 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-23 03:52 ` Ajin Cherian <itsajin@gmail.com>
2023-03-23 13:36 ` vignesh C <vignesh21@gmail.com>
2023-03-26 14:06 ` vignesh C <vignesh21@gmail.com>
2023-03-26 21:21 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-27 06:37 ` Amit Kapila <amit.kapila16@gmail.com>
2023-03-27 12:07 ` Amit Kapila <amit.kapila16@gmail.com>
2023-03-28 04:12 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-30 06:37 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-30 07:59 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-30 09:45 ` vignesh C <vignesh21@gmail.com>
2023-03-30 10:22 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-04 03:13 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-04-04 11:34 ` shveta malik <shveta.malik@gmail.com>
2023-04-07 03:12 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-04-07 03:22 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-04-10 09:46 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-11 09:02 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-10 11:20 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-17 12:02 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-20 03:41 ` shveta malik <shveta.malik@gmail.com>
2023-04-20 08:58 ` shveta malik <shveta.malik@gmail.com>
2023-04-20 12:39 ` shveta malik <shveta.malik@gmail.com>
2023-04-21 09:16 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-25 03:57 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-26 04:31 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-04-26 05:56 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-26 06:31 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-04-26 08:56 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-28 11:41 ` Amit Kapila <amit.kapila16@gmail.com>
2023-05-02 03:00 ` shveta malik <shveta.malik@gmail.com>
2023-05-08 10:28 ` shveta malik <shveta.malik@gmail.com>
2023-05-08 10:39 ` shveta malik <shveta.malik@gmail.com>
2023-05-08 11:01 ` shveta malik <shveta.malik@gmail.com>
2023-05-09 10:53 ` shveta malik <shveta.malik@gmail.com>
2023-05-12 06:33 ` shveta malik <shveta.malik@gmail.com>
2023-05-17 10:11 ` shveta malik <shveta.malik@gmail.com>
2023-05-17 11:15 ` vignesh C <vignesh21@gmail.com>
2023-05-22 05:56 ` shveta malik <shveta.malik@gmail.com>
2023-05-24 05:38 ` vignesh C <vignesh21@gmail.com>
2023-05-29 12:46 ` vignesh C <vignesh21@gmail.com>
2023-05-30 02:53 ` shveta malik <shveta.malik@gmail.com>
2023-05-30 11:19 ` vignesh C <vignesh21@gmail.com>
2023-05-31 09:02 ` Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
2023-06-01 15:42 ` vignesh C <vignesh21@gmail.com>
2023-06-06 06:01 ` Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
2023-06-08 12:18 ` shveta malik <shveta.malik@gmail.com>
2023-05-29 06:15 ` Yu Shi (Fujitsu) <shiy.fnst@fujitsu.com>
2023-05-31 09:40 ` shveta malik <shveta.malik@gmail.com>
2023-06-01 02:12 ` Yu Shi (Fujitsu) <shiy.fnst@fujitsu.com>
2023-06-06 09:12 ` vignesh C <vignesh21@gmail.com>
2023-06-01 07:39 ` Yu Shi (Fujitsu) <shiy.fnst@fujitsu.com>
2023-06-05 09:30 ` shveta malik <shveta.malik@gmail.com>
2023-06-06 10:56 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-08 05:05 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-08 12:01 ` shveta malik <shveta.malik@gmail.com>
2023-06-09 00:05 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-06-09 05:38 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-09 03:10 ` shveta malik <shveta.malik@gmail.com>
2023-06-09 09:51 ` Ajin Cherian <itsajin@gmail.com>
2023-06-12 01:47 ` Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
2023-06-13 07:50 ` Michael Paquier <michael@paquier.xyz>
2023-06-13 13:19 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-13 21:56 ` Michael Paquier <michael@paquier.xyz>
2023-06-16 10:31 ` shveta malik <shveta.malik@gmail.com>
2023-06-19 03:39 ` shveta malik <shveta.malik@gmail.com>
2023-06-19 09:34 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-21 13:08 ` Jelte Fennema <postgres@jeltef.nl>
2023-06-22 10:45 ` shveta malik <shveta.malik@gmail.com>
2023-06-22 04:09 ` shveta malik <shveta.malik@gmail.com>
2023-06-30 07:06 ` shveta malik <shveta.malik@gmail.com>
2023-07-11 11:01 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-07-14 10:33 ` Amit Kapila <amit.kapila16@gmail.com>
2023-07-18 05:28 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-07-18 05:37 ` Michael Paquier <michael@paquier.xyz>
2023-07-18 07:09 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2024-03-28 12:00 ` Andrey M. Borodin <x4mmm@yandex-team.ru>
2024-03-28 12:03 ` Amit Kapila <amit.kapila16@gmail.com>
2024-07-25 05:42 ` Konstantin Berkaev <konstantinberkaev@gmail.com>
2026-02-02 16:04 ` Vitaly Davydov <v.davydov@postgrespro.ru>
2026-02-04 07:39 ` Masahiko Sawada <sawada.mshk@gmail.com>
2026-02-04 11:44 ` Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
2026-02-09 10:42 ` Vitaly Davydov <v.davydov@postgrespro.ru>
2026-02-24 01:21 ` Masahiko Sawada <sawada.mshk@gmail.com>
2026-02-12 22:12 ` Bruce Momjian <bruce@momjian.us>
2026-02-16 09:54 ` Amit Kapila <amit.kapila16@gmail.com>
2026-02-16 15:16 ` Bruce Momjian <bruce@momjian.us>
2026-02-09 13:40 ` Andreas Karlsson <andreas@proxel.se>
2026-02-20 15:07 ` Vitaly Davydov <v.davydov@postgrespro.ru>
2023-06-22 10:52 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-27 10:15 ` vignesh C <vignesh21@gmail.com>
2023-07-09 19:22 ` Zheng Li <zhengli10@gmail.com>
2023-07-10 06:04 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-06-22 04:12 ` shveta malik <shveta.malik@gmail.com>
2023-06-13 05:15 ` Amit Kapila <amit.kapila16@gmail.com>
2023-06-08 12:07 ` shveta malik <shveta.malik@gmail.com>
2023-05-03 09:04 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-04-21 05:57 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-20 10:10 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-24 02:48 ` shveta malik <shveta.malik@gmail.com>
2023-04-24 03:46 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-11 04:05 ` Yu Shi (Fujitsu) <shiy.fnst@fujitsu.com>
2023-04-11 04:19 ` Wei Wang (Fujitsu) <wangw.fnst@fujitsu.com>
2023-04-12 11:23 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-13 09:31 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-17 12:02 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-14 07:36 ` vignesh C <vignesh21@gmail.com>
2023-04-15 01:08 ` vignesh C <vignesh21@gmail.com>
2023-04-17 10:31 ` vignesh C <vignesh21@gmail.com>
2023-04-19 13:55 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-20 01:49 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-04-17 08:14 ` Peter Smith <smithpb2250@gmail.com>
2023-04-17 12:16 ` Peter Smith <smithpb2250@gmail.com>
2023-04-19 08:27 ` Peter Smith <smithpb2250@gmail.com>
2023-04-21 00:26 ` Peter Smith <smithpb2250@gmail.com>
2023-05-02 02:23 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-05-05 10:09 ` Peter Smith <smithpb2250@gmail.com>
2023-03-30 11:08 ` vignesh C <vignesh21@gmail.com>
2023-04-02 09:55 ` Phil Florent <philflorent@hotmail.com>
2023-04-03 04:07 ` Amit Kapila <amit.kapila16@gmail.com>
2023-04-03 13:25 ` Phil Florent <philflorent@hotmail.com>
2023-03-30 22:30 ` Peter Smith <smithpb2250@gmail.com>
2023-04-04 03:21 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-03-28 05:40 ` Amit Kapila <amit.kapila16@gmail.com>
2023-03-28 06:22 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-04-26 06:31 ` Masahiko Sawada <sawada.mshk@gmail.com>
2023-04-26 09:33 ` vignesh C <vignesh21@gmail.com>
2023-04-26 11:31 ` Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>
2023-03-28 14:51 ` Jonathan S. Katz <jkatz@postgresql.org>
2023-03-28 21:19 ` Zheng Li <zhengli10@gmail.com>
2023-03-29 09:13 ` Amit Kapila <amit.kapila16@gmail.com>
2023-03-29 16:53 ` Zheng Li <zhengli10@gmail.com>
2023-03-30 10:31 ` Amit Kapila <amit.kapila16@gmail.com>
2023-03-26 12:38 ` vignesh C <vignesh21@gmail.com>
2023-03-27 03:57 ` vignesh C <vignesh21@gmail.com>
2023-03-17 14:49 ` Takamichi Osumi (Fujitsu) <osumi.takamichi@fujitsu.com>
2023-03-17 15:40 ` Zheng Li <zhengli10@gmail.com>
2023-02-06 01:16 ` Peter Smith <smithpb2250@gmail.com>
2023-02-06 11:32 ` vignesh C <vignesh21@gmail.com>
2023-02-07 12:05 ` vignesh C <vignesh21@gmail.com>
2023-02-10 16:24 ` vignesh C <vignesh21@gmail.com>
2023-02-23 11:42 ` Ajin Cherian <itsajin@gmail.com>
2023-02-28 04:24 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-08 22:16 ` Peter Smith <smithpb2250@gmail.com>
2023-02-10 16:20 ` vignesh C <vignesh21@gmail.com>
2023-02-10 16:31 ` vignesh C <vignesh21@gmail.com>
2023-02-13 04:20 ` Zheng Li <zhengli10@gmail.com>
2023-02-13 04:55 ` Peter Smith <smithpb2250@gmail.com>
2023-02-15 06:50 ` Amit Kapila <amit.kapila16@gmail.com>
2023-02-14 01:43 ` Peter Smith <smithpb2250@gmail.com>
2023-02-14 09:27 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-02-14 10:28 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2023-02-15 04:00 ` Peter Smith <smithpb2250@gmail.com>
2023-01-18 11:26 ` Amit Kapila <amit.kapila16@gmail.com>
2023-01-19 03:09 ` Zheng Li <zhengli10@gmail.com>
2023-01-19 07:05 ` Amit Kapila <amit.kapila16@gmail.com>
2023-01-19 17:54 ` Zheng Li <zhengli10@gmail.com>
2023-01-23 08:46 ` Amit Kapila <amit.kapila16@gmail.com>
2023-01-24 03:27 ` Zheng Li <zhengli10@gmail.com>
2022-12-27 22:32 ` Zheng Li <zhengli10@gmail.com>
2022-12-13 09:17 ` vignesh C <vignesh21@gmail.com>
2022-12-19 13:19 ` Ajin Cherian <itsajin@gmail.com>
2022-12-19 15:29 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-12-21 00:02 ` Peter Smith <smithpb2250@gmail.com>
2022-11-17 19:01 ` Zheng Li <zhengli10@gmail.com>
2022-11-14 23:57 ` Peter Smith <smithpb2250@gmail.com>
2022-12-12 12:20 ` Ajin Cherian <itsajin@gmail.com>
2022-12-12 13:13 ` Ajin Cherian <itsajin@gmail.com>
2022-12-12 17:40 ` Zheng Li <zhengli10@gmail.com>
2022-12-12 22:49 ` Zheng Li <zhengli10@gmail.com>
2022-12-13 02:37 ` li jie <ggysxcq@gmail.com>
2022-12-13 22:25 ` Zheng Li <zhengli10@gmail.com>
2022-12-14 10:34 ` vignesh C <vignesh21@gmail.com>
2022-12-15 05:36 ` Zheng Li <zhengli10@gmail.com>
2022-08-04 02:51 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-07-01 16:52 ` vignesh C <vignesh21@gmail.com>
2022-07-02 03:21 ` Amit Kapila <amit.kapila16@gmail.com>
2022-07-02 14:24 ` vignesh C <vignesh21@gmail.com>
2022-06-28 16:47 ` vignesh C <vignesh21@gmail.com>
2022-06-22 22:22 ` Zheng Li <zhengli10@gmail.com>
2022-06-23 07:09 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-22 05:38 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-06-23 10:00 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-24 02:39 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-06-24 10:19 ` Amit Kapila <amit.kapila16@gmail.com>
2022-06-15 06:45 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-06-23 12:43 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-05-27 16:15 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-03-18 00:22 ` Japin Li <japinli@hotmail.com>
2022-03-18 00:38 ` Japin Li <japinli@hotmail.com>
2022-04-08 11:13 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-08 11:34 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-04-08 13:12 ` Robert Haas <robertmhaas@gmail.com>
2022-04-11 05:04 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-06 13:24 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-06 16:51 ` Zheng Li <zhengli10@gmail.com>
2022-05-07 04:08 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-08 07:09 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-05-09 07:16 ` Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
2022-05-09 08:56 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-09 09:05 ` Alvaro Herrera <alvherre@alvh.no-ip.org>
2022-05-10 15:48 ` Zheng Li <zhengli10@gmail.com>
2022-05-11 04:45 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-05-11 07:38 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-05-11 09:15 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-11 12:55 ` Masahiko Sawada <sawada.mshk@gmail.com>
2022-05-12 02:54 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-10 11:33 ` Ajin Cherian <itsajin@gmail.com>
2022-05-10 12:39 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-10 15:17 ` Ajin Cherian <itsajin@gmail.com>
2022-05-11 06:21 ` Ajin Cherian <itsajin@gmail.com>
2022-05-26 05:45 ` Ajin Cherian <itsajin@gmail.com>
2022-05-27 09:07 ` houzj.fnst@fujitsu.com <houzj.fnst@fujitsu.com>
2022-03-17 00:08 ` Japin Li <japinli@hotmail.com>
2022-03-17 15:56 ` Japin Li <japinli@hotmail.com>
2022-03-21 08:13 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-03-24 10:02 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-03-24 11:36 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-03-24 17:54 ` Zheng Li <zhengli10@gmail.com>
2022-03-29 04:17 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-04-07 10:19 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-08 11:00 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-14 09:26 ` Dilip Kumar <dilipbalaut@gmail.com>
2022-04-14 14:15 ` Euler Taveira <euler@eulerto.com>
2022-04-14 20:31 ` Zheng Li <zhengli10@gmail.com>
2022-04-29 17:34 ` Zheng Li <zhengli10@gmail.com>
2022-05-23 11:54 ` Amit Kapila <amit.kapila16@gmail.com>
2022-05-23 15:40 ` Zheng Li <zhengli10@gmail.com>
2022-04-11 16:42 ` Zheng Li <zhengli10@gmail.com>
2022-04-13 00:19 ` Zheng Li <zhengli10@gmail.com>
2022-04-13 03:47 ` Amit Kapila <amit.kapila16@gmail.com>
2022-04-15 02:08 ` Zheng Li <zhengli10@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox