agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
25+ messages / 3 participants
[nested] [flat]
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v8 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index c4fe75df8e..a8bbfcc71f 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -683,7 +683,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 71de7cf80e..e6bb27aade 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--AZuoSAvZwvV/ife4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v8-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v10 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 5b8a1e4b61..f36c29ce7f 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -686,7 +686,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--8nsIa27JVQLqB7/C
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="v10-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)
..rather than line length
---
src/backend/commands/copyfrom.c | 2 +-
src/include/executor/nodeModifyTable.h | 6 +++---
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
/* Add this tuple to the tuple buffer */
MultiInsertInfoStore(&cstate->miinfo,
resultRelInfo, myslot,
- cstate->line_buf.len,
+ MemoryContextMemAllocated(myslot->tts_mcxt, true),
cstate->miinfo.cur_lineno);
/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
#define MAX_BUFFERED_TUPLES 1000
/*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
*/
-#define MAX_BUFFERED_BYTES 65535
+#define MAX_BUFFERED_BYTES (1024*1024*8)
/* Trim the list of buffers back down to this number after flushing */
#define MAX_PARTITION_BUFFERS 32
--
2.17.0
--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: bogus: logical replication rows/cols combinations
@ 2022-04-27 09:53 Alvaro Herrera <[email protected]>
2022-04-28 12:13 ` Re: bogus: logical replication rows/cols combinations Peter Eisentraut <[email protected]>
0 siblings, 1 reply; 25+ messages in thread
From: Alvaro Herrera @ 2022-04-27 09:53 UTC (permalink / raw)
To: Amit Kapila <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Pg Hackers <[email protected]>
On 2022-Apr-27, Amit Kapila wrote:
> On Tue, Apr 26, 2022 at 4:00 AM Tomas Vondra
> <[email protected]> wrote:
> > I can take a stab at it, but it seems strange to not apply the same
> > logic to evaluation of publish_as_relid.
>
> Yeah, the current behavior seems to be consistent with what we already
> do.
Sorry, this argument makes no sense to me. The combination of both
features is not consistent, and both features are new.
'publish_as_relid' is an implementation detail. If the implementation
fails to follow the feature design, then the implementation must be
fixed ... not the design!
IMO, we should first determine how we want row filters and column lists
to work when used in conjunction -- for relations (sets of rows) in a
general sense. After we have done that, then we can use that design to
drive how we want partitioned tables to be handled for it. Keep in mind
that when users see a partitioned table, what they first see is a table.
They want all their tables to work in pretty much the same way --
partitioned or not partitioned. The fact that a table is partitioned
should affect as little as possible the way it interacts with other
features.
Now, another possibility is to say "naah, this is too hard", or even
"naah, there's no time to write all that for this release". That might
be okay, but in that case let's add an implementation restriction to
ensure that we don't paint ourselves in a corner regarding what is
reasonable behavior. For example, an easy restriction might be: if a
table is in multiple publications with mismatching row filters/column
lists, then a subscriber is not allowed to subscribe to both
publications. (Maybe this restriction isn't exactly what we need so
that it actually implements what we need, not sure). Then, if/when in
the future we implement this correctly, we can lift the restriction.
--
Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/
"La conclusión que podemos sacar de esos estudios es que
no podemos sacar ninguna conclusión de ellos" (Tanenbaum)
^ permalink raw reply [nested|flat] 25+ messages in thread
* Re: bogus: logical replication rows/cols combinations
2022-04-27 09:53 Re: bogus: logical replication rows/cols combinations Alvaro Herrera <[email protected]>
@ 2022-04-28 12:13 ` Peter Eisentraut <[email protected]>
0 siblings, 0 replies; 25+ messages in thread
From: Peter Eisentraut @ 2022-04-28 12:13 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; Amit Kapila <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Pg Hackers <[email protected]>
On 27.04.22 11:53, Alvaro Herrera wrote:
> Now, another possibility is to say "naah, this is too hard", or even
> "naah, there's no time to write all that for this release". That might
> be okay, but in that case let's add an implementation restriction to
> ensure that we don't paint ourselves in a corner regarding what is
> reasonable behavior. For example, an easy restriction might be: if a
> table is in multiple publications with mismatching row filters/column
> lists, then a subscriber is not allowed to subscribe to both
> publications. (Maybe this restriction isn't exactly what we need so
> that it actually implements what we need, not sure). Then, if/when in
> the future we implement this correctly, we can lift the restriction.
My feeling is also that we should prohibit the combinations that we
cannot make work correctly.
^ permalink raw reply [nested|flat] 25+ messages in thread
end of thread, other threads:[~2022-04-28 12:13 UTC | newest]
Thread overview: 25+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v8 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v10 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2022-04-27 09:53 Re: bogus: logical replication rows/cols combinations Alvaro Herrera <[email protected]>
2022-04-28 12:13 ` Re: bogus: logical replication rows/cols combinations Peter Eisentraut <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox