agora inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
36+ messages / 2 participants
[nested] [flat]

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH v5 3/7] Row pattern recognition patch (planner).
@ 2023-09-02 06:32 Tatsuo Ishii <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: Tatsuo Ishii @ 2023-09-02 06:32 UTC (permalink / raw)

---
 src/backend/optimizer/plan/createplan.c | 23 ++++++++++++++++++-----
 src/backend/optimizer/plan/setrefs.c    | 23 +++++++++++++++++++++++
 src/include/nodes/plannodes.h           | 15 +++++++++++++++
 3 files changed, 56 insertions(+), 5 deletions(-)

diff --git a/src/backend/optimizer/plan/createplan.c b/src/backend/optimizer/plan/createplan.c
index 34ca6d4ac2..e3c07ded65 100644
--- a/src/backend/optimizer/plan/createplan.c
+++ b/src/backend/optimizer/plan/createplan.c
@@ -286,9 +286,10 @@ static WindowAgg *make_windowagg(List *tlist, Index winref,
 								 int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations,
 								 int frameOptions, Node *startOffset, Node *endOffset,
 								 Oid startInRangeFunc, Oid endInRangeFunc,
-								 Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst,
-								 List *runCondition, List *qual, bool topWindow,
-								 Plan *lefttree);
+								 Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition,
+								 RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp, List *defineClause,
+								 List *defineInitial,
+								 List *qual, bool topWindow, Plan *lefttree);
 static Group *make_group(List *tlist, List *qual, int numGroupCols,
 						 AttrNumber *grpColIdx, Oid *grpOperators, Oid *grpCollations,
 						 Plan *lefttree);
@@ -2698,6 +2699,11 @@ create_windowagg_plan(PlannerInfo *root, WindowAggPath *best_path)
 						  wc->inRangeAsc,
 						  wc->inRangeNullsFirst,
 						  wc->runCondition,
+						  wc->rpSkipTo,
+						  wc->patternVariable,
+						  wc->patternRegexp,
+						  wc->defineClause,
+						  wc->defineInitial,
 						  best_path->qual,
 						  best_path->topwindow,
 						  subplan);
@@ -6601,8 +6607,10 @@ make_windowagg(List *tlist, Index winref,
 			   int ordNumCols, AttrNumber *ordColIdx, Oid *ordOperators, Oid *ordCollations,
 			   int frameOptions, Node *startOffset, Node *endOffset,
 			   Oid startInRangeFunc, Oid endInRangeFunc,
-			   Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst,
-			   List *runCondition, List *qual, bool topWindow, Plan *lefttree)
+			   Oid inRangeColl, bool inRangeAsc, bool inRangeNullsFirst, List *runCondition, 
+			   RPSkipTo rpSkipTo, List *patternVariable, List *patternRegexp, List *defineClause,
+			   List *defineInitial,
+			   List *qual, bool topWindow, Plan *lefttree)
 {
 	WindowAgg  *node = makeNode(WindowAgg);
 	Plan	   *plan = &node->plan;
@@ -6628,6 +6636,11 @@ make_windowagg(List *tlist, Index winref,
 	node->inRangeAsc = inRangeAsc;
 	node->inRangeNullsFirst = inRangeNullsFirst;
 	node->topWindow = topWindow;
+	node->rpSkipTo = rpSkipTo,
+	node->patternVariable = patternVariable;
+	node->patternRegexp = patternRegexp;
+	node->defineClause = defineClause;
+	node->defineInitial = defineInitial;
 
 	plan->targetlist = tlist;
 	plan->lefttree = lefttree;
diff --git a/src/backend/optimizer/plan/setrefs.c b/src/backend/optimizer/plan/setrefs.c
index 97fa561e4e..2ed00b5d41 100644
--- a/src/backend/optimizer/plan/setrefs.c
+++ b/src/backend/optimizer/plan/setrefs.c
@@ -2456,6 +2456,29 @@ set_upper_references(PlannerInfo *root, Plan *plan, int rtoffset)
 					   NRM_EQUAL,
 					   NUM_EXEC_QUAL(plan));
 
+	/*
+	 *	Modifies an expression tree in each DEFINE clause so that all Var
+	 *	nodes reference outputs of a subplan.
+	 */
+	if (IsA(plan, WindowAgg))
+	{
+		WindowAgg  *wplan = (WindowAgg *) plan;
+
+		foreach(l, wplan->defineClause)
+		{
+			TargetEntry *tle = (TargetEntry *) lfirst(l);
+
+			tle->expr = (Expr *)
+				fix_upper_expr(root,
+							   (Node *) tle->expr,
+							   subplan_itlist,
+							   OUTER_VAR,
+							   rtoffset,
+							   NRM_EQUAL,
+							   NUM_EXEC_QUAL(plan));
+		}
+	}
+
 	pfree(subplan_itlist);
 }
 
diff --git a/src/include/nodes/plannodes.h b/src/include/nodes/plannodes.h
index 1b787fe031..19815a98bb 100644
--- a/src/include/nodes/plannodes.h
+++ b/src/include/nodes/plannodes.h
@@ -1096,6 +1096,21 @@ typedef struct WindowAgg
 	/* nulls sort first for in_range tests? */
 	bool		inRangeNullsFirst;
 
+	/* Row Pattern Recognition AFTER MACH SKIP clause */
+	RPSkipTo	rpSkipTo;		/* Row Pattern Skip To type */	
+
+	/* Row Pattern PATTERN variable name (list of String) */
+	List		*patternVariable;
+
+	/* Row Pattern RPATTERN regular expression quantifier ('+' or ''. list of String) */
+	List		*patternRegexp;
+
+	/* Row Pattern DEFINE clause (list of TargetEntry) */
+	List	   *defineClause;
+
+	/* Row Pattern DEFINE variable initial names (list of String) */
+	List		*defineInitial;
+
 	/*
 	 * false for all apart from the WindowAgg that's closest to the root of
 	 * the plan
-- 
2.25.1


----Next_Part(Sat_Sep__2_15_52_35_2023_273)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="v5-0004-Row-pattern-recognition-patch-executor.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

end of thread, other threads:[~2023-09-02 06:32 UTC | newest]

Thread overview: 36+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2023-09-02 06:32 [PATCH v5 3/7] Row pattern recognition patch (planner). Tatsuo Ishii <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox