agora inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
36+ messages / 2 participants
[nested] [flat]

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi, Ragnar Ouchterlony
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
         https://www.postgresql.org/message-id/16d73804-c9cd-14c5-463e-5caad563ff77%40agama.tv
---
 src/backend/commands/copy.c | 18 +++++++++++++-----
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..9624c93f6b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,14 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind != RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
+
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2708,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------36267A5A88CA9EA4AEC9E6E5
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables
@ 2016-12-28 01:28 amit <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: amit @ 2016-12-28 01:28 UTC (permalink / raw)

We might consider alleviating this restriction by allocating a
BulkInsertState object and managing tuple-buffering per partition.

Reported by: 高增琦, Venkata B Nagothi
Patch by: Amit Langote (with pointers from 高增琦)
Reports: https://www.postgresql.org/message-id/CAFmBtr32FDOqofo8yG-4mjzL1HnYHxXK5S9OGFJ%3D%3DcJpgEW4vA%40mail...
         https://www.postgresql.org/message-id/CAEyp7J9WiX0L3DoiNcRrY-9iyw%3DqP%2Bj%3DDLsAnNFF1xT2J1ggfQ%40ma...
---
 src/backend/commands/copy.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f56b2ac49b..322f4260fd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2296,7 +2296,7 @@ CopyFrom(CopyState cstate)
 	ErrorContextCallback errcallback;
 	CommandId	mycid = GetCurrentCommandId(true);
 	int			hi_options = 0; /* start with default heap_insert options */
-	BulkInsertState bistate;
+	BulkInsertState bistate = NULL;
 	uint64		processed = 0;
 	bool		useHeapMultiInsert;
 	int			nBufferedTuples = 0;
@@ -2455,8 +2455,8 @@ CopyFrom(CopyState cstate)
 	if ((resultRelInfo->ri_TrigDesc != NULL &&
 		 (resultRelInfo->ri_TrigDesc->trig_insert_before_row ||
 		  resultRelInfo->ri_TrigDesc->trig_insert_instead_row)) ||
-		cstate->partition_dispatch_info != NULL ||
-		cstate->volatile_defexprs)
+		cstate->volatile_defexprs ||
+		cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
 	{
 		useHeapMultiInsert = false;
 	}
@@ -2480,7 +2480,13 @@ CopyFrom(CopyState cstate)
 	values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
 	nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
 
-	bistate = GetBulkInsertState();
+	/*
+	 * FIXME: We don't engage the bulk-insert mode for partitioned tables,
+	 * because the the heap relation is most likely change from one row to
+	 * next due to tuple-routing.
+	 */
+	if (cstate->rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+		bistate = GetBulkInsertState();
 	econtext = GetPerTupleExprContext(estate);
 
 	/* Set up callback to identify error line number */
@@ -2701,7 +2707,8 @@ CopyFrom(CopyState cstate)
 	/* Done, clean up */
 	error_context_stack = errcallback.previous;
 
-	FreeBulkInsertState(bistate);
+	if (bistate != NULL)
+		FreeBulkInsertState(bistate);
 
 	MemoryContextSwitchTo(oldcontext);
 
-- 
2.11.0


--------------35DA5B82994CF4AD40E20FE2
Content-Type: text/x-diff;
 name="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename*0="0003-Support-bulk-insert-mode-for-partitioned-tables.patch"



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

* [PATCH v20240912 1/4] Refactor float8out_internval for better performance
@ 2024-09-11 04:19 Andy Fan <[email protected]>
  0 siblings, 0 replies; 36+ messages in thread

From: Andy Fan @ 2024-09-11 04:19 UTC (permalink / raw)

Some users like cube, geo needs calls float8out_internval to get a
string, and then copy them into its own StringInfo. In this commit,
we would let the user provide a buffer to float8out_internal so that it
can put the data to buffer directly. This commit also reuse the existing
string length to avoid another strlen call in appendStringInfoString.
---
 contrib/cube/cube.c             | 17 +++++++++++++++--
 src/backend/utils/adt/float.c   | 14 ++++++++------
 src/backend/utils/adt/geo_ops.c | 34 ++++++++++++++++++++++-----------
 src/include/catalog/pg_type.h   |  1 +
 src/include/utils/float.h       |  2 +-
 5 files changed, 48 insertions(+), 20 deletions(-)

diff --git a/contrib/cube/cube.c b/contrib/cube/cube.c
index 1fc447511a..a239acf35c 100644
--- a/contrib/cube/cube.c
+++ b/contrib/cube/cube.c
@@ -12,6 +12,7 @@
 
 #include "access/gist.h"
 #include "access/stratnum.h"
+#include "catalog/pg_type.h"
 #include "cubedata.h"
 #include "libpq/pqformat.h"
 #include "utils/array.h"
@@ -295,26 +296,38 @@ cube_out(PG_FUNCTION_ARGS)
 	StringInfoData buf;
 	int			dim = DIM(cube);
 	int			i;
+	int str_len;
 
 	initStringInfo(&buf);
 
 	appendStringInfoChar(&buf, '(');
+
+	/* 3 for ", " and 1 for '\0'. */
+	enlargeStringInfo(&buf, (MAXFLOAT8LEN + 4) * dim);
 	for (i = 0; i < dim; i++)
 	{
 		if (i > 0)
 			appendStringInfoString(&buf, ", ");
-		appendStringInfoString(&buf, float8out_internal(LL_COORD(cube, i)));
+		float8out_internal(LL_COORD(cube, i), buf.data + buf.len, &str_len);
+		buf.len += str_len;
+		buf.data[buf.len] = '\0';
 	}
 	appendStringInfoChar(&buf, ')');
 
 	if (!cube_is_point_internal(cube))
 	{
 		appendStringInfoString(&buf, ",(");
+
+		/* 3 for ", " and 1 for '\0'. */
+		enlargeStringInfo(&buf, (MAXFLOAT8LEN + 4) * dim);
 		for (i = 0; i < dim; i++)
 		{
 			if (i > 0)
 				appendStringInfoString(&buf, ", ");
-			appendStringInfoString(&buf, float8out_internal(UR_COORD(cube, i)));
+
+			float8out_internal(UR_COORD(cube, i), buf.data + buf.len, &str_len);
+			buf.len += str_len;
+			buf.data[buf.len] = '\0';
 		}
 		appendStringInfoChar(&buf, ')');
 	}
diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c
index f709c21e1f..1f31f8540e 100644
--- a/src/backend/utils/adt/float.c
+++ b/src/backend/utils/adt/float.c
@@ -531,22 +531,24 @@ float8out(PG_FUNCTION_ARGS)
  * float8out_internal - guts of float8out()
  *
  * This is exposed for use by functions that want a reasonably
- * platform-independent way of outputting doubles.
- * The result is always palloc'd.
+ * platform-independent way of outputting doubles, output the
+ * string length to *len;
  */
 char *
-float8out_internal(double num)
+float8out_internal(double num, char *ascii, int *len)
 {
-	char	   *ascii = (char *) palloc(32);
 	int			ndig = DBL_DIG + extra_float_digits;
 
+	if (ascii == NULL)
+		ascii = (char *) palloc(MAXFLOAT8LEN);
+
 	if (extra_float_digits > 0)
 	{
-		double_to_shortest_decimal_buf(num, ascii);
+		*len = double_to_shortest_decimal_buf(num, ascii);
 		return ascii;
 	}
 
-	(void) pg_strfromd(ascii, 32, ndig, num);
+	*len = pg_strfromd(ascii, 32, ndig, num);
 	return ascii;
 }
 
diff --git a/src/backend/utils/adt/geo_ops.c b/src/backend/utils/adt/geo_ops.c
index 07d1649c7b..59f2feaa59 100644
--- a/src/backend/utils/adt/geo_ops.c
+++ b/src/backend/utils/adt/geo_ops.c
@@ -29,6 +29,7 @@
 #include <float.h>
 #include <ctype.h>
 
+#include "catalog/pg_type.h"
 #include "libpq/pqformat.h"
 #include "miscadmin.h"
 #include "nodes/miscnodes.h"
@@ -202,10 +203,12 @@ single_decode(char *num, float8 *x, char **endptr_p,
 static void
 single_encode(float8 x, StringInfo str)
 {
-	char	   *xstr = float8out_internal(x);
+	int str_len;
+	enlargeStringInfo(str, MAXFLOAT8LEN + 1);
+	float8out_internal(x, str->data + str->len, &str_len);
 
-	appendStringInfoString(str, xstr);
-	pfree(xstr);
+	str->len += str_len;
+	str->data[str->len] = '\0';
 }								/* single_encode() */
 
 static bool
@@ -254,12 +257,20 @@ fail:
 static void
 pair_encode(float8 x, float8 y, StringInfo str)
 {
-	char	   *xstr = float8out_internal(x);
-	char	   *ystr = float8out_internal(y);
+	int data_len;
+	/* the additional 2 is for ',' and '\0' */
+	enlargeStringInfo(str, MAXFLOAT8LEN * 2 + 2);
 
-	appendStringInfo(str, "%s,%s", xstr, ystr);
-	pfree(xstr);
-	pfree(ystr);
+	float8out_internal(x, str->data + str->len, &data_len);
+	str->len += data_len;
+
+	str->data[str->len] = ',';
+	str->len++;
+
+	float8out_internal(y, str->data + str->len, &data_len);
+	str->len += data_len;
+
+	str->data[str->len] = '\0';
 }
 
 static bool
@@ -1023,9 +1034,10 @@ Datum
 line_out(PG_FUNCTION_ARGS)
 {
 	LINE	   *line = PG_GETARG_LINE_P(0);
-	char	   *astr = float8out_internal(line->A);
-	char	   *bstr = float8out_internal(line->B);
-	char	   *cstr = float8out_internal(line->C);
+	int datalen;
+	char	   *astr = float8out_internal(line->A, NULL, &datalen);
+	char	   *bstr = float8out_internal(line->B, NULL, &datalen);
+	char	   *cstr = float8out_internal(line->C, NULL, &datalen);
 
 	PG_RETURN_CSTRING(psprintf("%c%s%c%s%c%s%c", LDELIM_L, astr, DELIM, bstr,
 							   DELIM, cstr, RDELIM_L));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e925969732..1ab8e4e4e9 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -344,6 +344,7 @@ MAKE_SYSCACHE(TYPENAMENSP, pg_type_typname_nsp_index, 64);
 
 #endif							/* EXPOSE_TO_CLIENT_CODE */
 
+#define MAXFLOAT8LEN 32
 
 extern ObjectAddress TypeShellMake(const char *typeName,
 								   Oid typeNamespace,
diff --git a/src/include/utils/float.h b/src/include/utils/float.h
index 7d1badd292..65c395299d 100644
--- a/src/include/utils/float.h
+++ b/src/include/utils/float.h
@@ -47,7 +47,7 @@ extern float8 float8in_internal(char *num, char **endptr_p,
 extern float4 float4in_internal(char *num, char **endptr_p,
 								const char *type_name, const char *orig_string,
 								struct Node *escontext);
-extern char *float8out_internal(float8 num);
+extern char *float8out_internal(float8 num, char *ascii, int *len);
 extern int	float4_cmp_internal(float4 a, float4 b);
 extern int	float8_cmp_internal(float8 a, float8 b);
 
-- 
2.45.1


--=-=-=
Content-Type: text/x-diff
Content-Disposition: attachment;
 filename=v20240912-0002-Continue-to-remove-some-unnecesary-strlen-.patch



^ permalink  raw  reply  [nested|flat] 36+ messages in thread

end of thread, other threads:[~2024-09-11 04:19 UTC | newest]

Thread overview: 36+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2016-12-28 01:28 [PATCH 2/3] No multi-insert and bulk-insert when COPYing into partitioned tables amit <[email protected]>
2024-09-11 04:19 [PATCH v20240912 1/4] Refactor float8out_internval for better performance Andy Fan <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox