public inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
24+ messages / 2 participants
[nested] [flat]

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v8 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index c4fe75df8e..a8bbfcc71f 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -683,7 +683,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 71de7cf80e..e6bb27aade 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--AZuoSAvZwvV/ife4
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v8-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 8221a2c5d3..e5ea909dfe 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -685,7 +685,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--3yNHWXBV/QO9xKNm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="v9-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* [PATCH v10 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples..
@ 2020-12-05 14:52  Justin Pryzby <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Justin Pryzby @ 2020-12-05 14:52 UTC (permalink / raw)

..rather than line length
---
 src/backend/commands/copyfrom.c        | 2 +-
 src/include/executor/nodeModifyTable.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/backend/commands/copyfrom.c b/src/backend/commands/copyfrom.c
index 5b8a1e4b61..f36c29ce7f 100644
--- a/src/backend/commands/copyfrom.c
+++ b/src/backend/commands/copyfrom.c
@@ -686,7 +686,7 @@ CopyFrom(CopyFromState cstate)
 					/* Add this tuple to the tuple buffer */
 					MultiInsertInfoStore(&cstate->miinfo,
 											 resultRelInfo, myslot,
-											 cstate->line_buf.len,
+											 MemoryContextMemAllocated(myslot->tts_mcxt, true),
 											 cstate->miinfo.cur_lineno);
 
 					/*
diff --git a/src/include/executor/nodeModifyTable.h b/src/include/executor/nodeModifyTable.h
index 30542a542a..87b689099c 100644
--- a/src/include/executor/nodeModifyTable.h
+++ b/src/include/executor/nodeModifyTable.h
@@ -41,10 +41,10 @@ extern void ExecReScanModifyTable(ModifyTableState *node);
 #define MAX_BUFFERED_TUPLES		1000
 
 /*
- * Flush buffers if there are >= this many bytes, as counted by the input
- * size, of tuples stored.
+ * Flush buffers if there are >= this many bytes of tuples stored, as counted
+ * by the slot's memory contexts.
  */
-#define MAX_BUFFERED_BYTES		65535
+#define MAX_BUFFERED_BYTES		(1024*1024*8)
 
 /* Trim the list of buffers back down to this number after flushing */
 #define MAX_PARTITION_BUFFERS	32
-- 
2.17.0


--8nsIa27JVQLqB7/C
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="v10-0004-WIP-check-tuple-size.patch"



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

* Re: MultiXact\SLRU buffers configuration
@ 2024-02-03 17:32  Andrey M. Borodin <[email protected]>
  0 siblings, 0 replies; 24+ messages in thread

From: Andrey M. Borodin @ 2024-02-03 17:32 UTC (permalink / raw)
  To: Alvaro Herrera <[email protected]>; +Cc: vignesh C <[email protected]>; Andrew Borodin <[email protected]>; [email protected]; Yura Sokolov <[email protected]>; Andres Freund <[email protected]>; Thomas Munro <[email protected]>; Gilles Darold <[email protected]>; Tomas Vondra <[email protected]>; Tomas Vondra <[email protected]>; Alexander Korotkov <[email protected]>; Anastasia Lubennikova <[email protected]>; Daniel Gustafsson <[email protected]>; Kyotaro Horiguchi <[email protected]>; pgsql-hackers



> On 28 Jan 2024, at 23:17, Andrey M. Borodin <[email protected]> wrote:
> 
> 
>> Perhaps a test to make the code reach the usleep(1000) can be written
>> using injection points (49cd2b93d7db)?
> 
> I've tried to prototype something like that. But interesting point between GetNewMultiXactId() and RecordNewMultiXact() is a critical section, and we cannot have injection points in critical sections...
> Also, to implement such a test we need "wait" type of injection points, see step 2 in attachment. With this type of injection points I can stop a backend amidst entering information about new MultiXact.

Here's the test draft. This test reliably reproduces sleep on CV when waiting next multixact to be filled into "members" SLRU.
Cost of having this test:
1. We need a new injection point type "wait" (in addition to "error" and "notice"). It cannot be avoided, because we need to sync at least 3 processed to observe condition we want.
2. We need new way to declare injection point that can happen inside critical section. I've called it "prepared injection point".

Complexity of having this test is higher than complexity of CV-sleep patch itself. Do we want it? If so I can produce cleaner version, currently all multixact tests are int injection_points test module.


Best regards, Andrey Borodin.


Attachments:

  [application/octet-stream] v2-0003-Test-multixact-CV-sleep.patch (9.1K, ../../[email protected]/2-v2-0003-Test-multixact-CV-sleep.patch)
  download | inline diff:
From 51ba2eadfd2a4f56130d026febb939ecdaa0ea91 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 28 Jan 2024 23:10:55 +0500
Subject: [PATCH v2 3/3] Test multixact CV sleep

---
 src/backend/access/transam/multixact.c        |  7 ++
 src/backend/utils/misc/injection_point.c      | 73 +++++++++++++++++++
 src/include/utils/injection_point.h           |  6 ++
 .../injection_points--1.0.sql                 | 11 +++
 .../injection_points/injection_points.c       | 29 ++++++++
 .../modules/injection_points/t/001_wait.pl    | 43 +++++++++++
 6 files changed, 169 insertions(+)

diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 03fcd25d4c..363a8bb1d7 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -89,6 +89,7 @@
 #include "storage/proc.h"
 #include "storage/procarray.h"
 #include "utils/builtins.h"
+#include "utils/injection_point.h"
 #include "utils/memutils.h"
 #include "utils/snapmgr.h"
 
@@ -822,8 +823,12 @@ MultiXactIdCreateFromMembers(int nmembers, MultiXactMember *members)
 	 * in vacuum.  During vacuum, in particular, it would be unacceptable to
 	 * keep OldestMulti set, in case it runs for long.
 	 */
+	INJECTION_POINT_PREPARE("GetNewMultiXactId-done");
+
 	multi = GetNewMultiXactId(nmembers, &offset);
 
+	INJECTION_POINT_RUN_PREPARED();
+
 	/* Make an XLOG entry describing the new MXID. */
 	xlrec.mid = multi;
 	xlrec.moff = offset;
@@ -1408,6 +1413,8 @@ retry:
 			LWLockRelease(MultiXactOffsetSLRULock);
 			CHECK_FOR_INTERRUPTS();
 
+			INJECTION_POINT("GetMultiXactIdMembers-CV-sleep");
+
 			ConditionVariableSleep(&MultiXactState->nextoff_cv,
 								   WAIT_EVENT_NEXT_MXMEMBERS);
 			ConditionVariableCancelSleep();
diff --git a/src/backend/utils/misc/injection_point.c b/src/backend/utils/misc/injection_point.c
index 398ef2cf30..c021889138 100644
--- a/src/backend/utils/misc/injection_point.c
+++ b/src/backend/utils/misc/injection_point.c
@@ -335,3 +335,76 @@ InjectionPointRun(const char *name)
 	elog(ERROR, "Injection points are not supported by this build");
 #endif
 }
+
+InjectionPointCallback prepared_injection_callback = NULL;
+const char* prepared_injection_callback_name;
+
+void
+InjectionPointPrepare(const char *name)
+{
+#ifdef USE_INJECTION_POINTS
+	InjectionPointEntry *entry_by_name;
+	bool		found;
+	Assert(prepared_injection_callback == NULL);
+
+	LWLockAcquire(InjectionPointLock, LW_SHARED);
+	entry_by_name = (InjectionPointEntry *)
+		hash_search(InjectionPointHash, name,
+					HASH_FIND, &found);
+	LWLockRelease(InjectionPointLock);
+
+	/*
+	 * If not found, do nothing and remove it from the local cache if it
+	 * existed there.
+	 */
+	if (!found)
+	{
+		injection_point_cache_remove(name);
+		return;
+	}
+
+	/*
+	 * Check if the callback exists in the local cache, to avoid unnecessary
+	 * external loads.
+	 */
+	prepared_injection_callback = injection_point_cache_get(name);
+	if (prepared_injection_callback == NULL)
+	{
+		char		path[MAXPGPATH];
+
+		/* not found in local cache, so load and register */
+		snprintf(path, MAXPGPATH, "%s/%s%s", pkglib_path,
+				 entry_by_name->library, DLSUFFIX);
+
+		if (!pg_file_exists(path))
+			elog(ERROR, "could not find library \"%s\" for injection point \"%s\"",
+				 path, name);
+
+		prepared_injection_callback = (InjectionPointCallback)
+			load_external_function(path, entry_by_name->function, false, NULL);
+
+		if (prepared_injection_callback == NULL)
+			elog(ERROR, "could not find function \"%s\" in library \"%s\" for injection point \"%s\"",
+				 entry_by_name->function, path, name);
+
+		/* add it to the local cache when found */
+		injection_point_cache_add(name, prepared_injection_callback);
+	}
+
+	prepared_injection_callback_name = name;
+#else
+	elog(ERROR, "Injection points are not supported by this build");
+#endif
+}
+
+void
+InjectionPointRunPrepared()
+{
+#ifdef USE_INJECTION_POINTS
+	if (prepared_injection_callback != NULL)
+		prepared_injection_callback(prepared_injection_callback_name);
+	prepared_injection_callback = NULL;
+#else
+	elog(ERROR, "Injection points are not supported by this build");
+#endif
+}
diff --git a/src/include/utils/injection_point.h b/src/include/utils/injection_point.h
index e07f6b7024..73d3aeac8a 100644
--- a/src/include/utils/injection_point.h
+++ b/src/include/utils/injection_point.h
@@ -16,8 +16,12 @@
  */
 #ifdef USE_INJECTION_POINTS
 #define INJECTION_POINT(name) InjectionPointRun(name)
+#define INJECTION_POINT_PREPARE(name) InjectionPointPrepare(name)
+#define INJECTION_POINT_RUN_PREPARED() InjectionPointRunPrepared()
 #else
 #define INJECTION_POINT(name) ((void) name)
+#define INJECTION_POINT_PREPARE(name) ((void) name)
+#define INJECTION_POINT_RUN_PREPARED()
 #endif
 
 /*
@@ -34,5 +38,7 @@ extern void InjectionPointAttach(const char *name,
 extern void InjectionPointRun(const char *name);
 extern void InjectionPointDetach(const char *name);
 extern bool InjectionPointIsAttach(const char *name);
+extern void InjectionPointPrepare(const char *name);
+extern void InjectionPointRunPrepared(void);
 
 #endif							/* INJECTION_POINT_H */
diff --git a/src/test/modules/injection_points/injection_points--1.0.sql b/src/test/modules/injection_points/injection_points--1.0.sql
index 5944c41716..d3ebda5964 100644
--- a/src/test/modules/injection_points/injection_points--1.0.sql
+++ b/src/test/modules/injection_points/injection_points--1.0.sql
@@ -33,3 +33,14 @@ CREATE FUNCTION injection_points_detach(IN point_name TEXT)
 RETURNS void
 AS 'MODULE_PATHNAME', 'injection_points_detach'
 LANGUAGE C STRICT PARALLEL UNSAFE;
+
+
+CREATE FUNCTION create_test_multixact()
+RETURNS xid
+AS 'MODULE_PATHNAME', 'create_test_multixact'
+LANGUAGE C STRICT PARALLEL UNSAFE;
+
+CREATE FUNCTION read_test_multixact(xid)
+RETURNS void
+AS 'MODULE_PATHNAME', 'read_test_multixact'
+LANGUAGE C STRICT PARALLEL UNSAFE;
\ No newline at end of file
diff --git a/src/test/modules/injection_points/injection_points.c b/src/test/modules/injection_points/injection_points.c
index fbb30b15ad..a0b9379886 100644
--- a/src/test/modules/injection_points/injection_points.c
+++ b/src/test/modules/injection_points/injection_points.c
@@ -109,3 +109,32 @@ injection_points_detach(PG_FUNCTION_ARGS)
 
 	PG_RETURN_VOID();
 }
+
+#include "access/multixact.h"
+#include "access/xact.h"
+
+PG_FUNCTION_INFO_V1(create_test_multixact);
+Datum
+create_test_multixact(PG_FUNCTION_ARGS)
+{
+	MultiXactId id;
+	MultiXactIdSetOldestMember();
+	id = MultiXactIdCreate(GetCurrentTransactionId(), MultiXactStatusUpdate,
+						GetCurrentTransactionId(), MultiXactStatusForShare);
+	PG_RETURN_TRANSACTIONID(id);
+}
+
+PG_FUNCTION_INFO_V1(read_test_multixact);
+Datum
+read_test_multixact(PG_FUNCTION_ARGS)
+{
+	MultiXactId id = PG_GETARG_TRANSACTIONID(0);
+	MultiXactMember *members;
+	INJECTION_POINT("read_test_multixact");
+	/* discard caches */
+	AtEOXact_MultiXact();
+
+	if (GetMultiXactIdMembers(id,&members,false, false) == -1)
+		elog(ERROR, "MultiXactId not found");
+	PG_RETURN_VOID();
+}
\ No newline at end of file
diff --git a/src/test/modules/injection_points/t/001_wait.pl b/src/test/modules/injection_points/t/001_wait.pl
index 98f7a8bcef..448dc97759 100644
--- a/src/test/modules/injection_points/t/001_wait.pl
+++ b/src/test/modules/injection_points/t/001_wait.pl
@@ -36,5 +36,48 @@ is($result, '0', 'wait injection point set');
 
 $bg->quit;
 
+# Test for Multixact generation edge case
+$node->safe_psql('postgres', q(select injection_points_attach('read_test_multixact','wait')));
+$node->safe_psql('postgres', q(select injection_points_attach('GetMultiXactIdMembers-CV-sleep','notice')));
+
+# This session must observe sleep on CV when generating multixact.
+# To achive this it first will create a multixact, then pause before reading it.
+my $observer = $node->background_psql('postgres');
+
+$observer->query_until(qr/start/,
+q(
+	\echo start
+	select read_test_multixact(create_test_multixact());
+));
+
+# This session will create next Multixact, it's necessary to avoid edge case 1 (see multixact.c)
+my $creator = $node->background_psql('postgres');
+$node->safe_psql('postgres', q(select injection_points_attach('GetNewMultiXactId-done','wait')));
+
+# We expect this query to hand in critical section after generating new multixact,
+# but before filling it's offset into SLRU
+$creator->query_until(qr/start/, q(
+	\echo start
+	select create_test_multixact();
+));
+
+# Now we are sure we can reach edge case 2. Proceed session that is reading that multixact.
+$node->safe_psql('postgres', q(select injection_points_detach('read_test_multixact')));
+
+# Release critical section. We have to do this so everyon can proceed.
+# But this is inherent race condition, I hope the tast will not be unstable here.
+# The only way to stabilize it will be adding some sleep here.
+$node->safe_psql('postgres', q(select injection_points_detach('GetNewMultiXactId-done')));
+
+# Here goes the whole purpose of this test: see that sleep in fact occured.
+ok( pump_until(
+		$observer->{run}, $observer->{timeout},
+		\$observer->{stderr}, qr/notice triggered for injection point GetMultiXactIdMembers-CV-sleep/),
+	"sleep observed");
+
+$observer->quit;
+
+$creator->quit;
+
 $node->stop;
 done_testing();
-- 
2.37.1 (Apple Git-137.1)



  [application/octet-stream] v2-0002-Add-wait-type-for-injection-points.patch (4.9K, ../../[email protected]/3-v2-0002-Add-wait-type-for-injection-points.patch)
  download | inline diff:
From 0924b7cc9f1a516df710677b246d26226cd01624 Mon Sep 17 00:00:00 2001
From: "Andrey M. Borodin" <[email protected]>
Date: Sun, 28 Jan 2024 22:22:22 +0500
Subject: [PATCH v2 2/3] Add wait type for injection points

---
 src/backend/utils/misc/injection_point.c      | 20 ++++++++++
 src/include/utils/injection_point.h           |  1 +
 src/test/modules/injection_points/Makefile    |  1 +
 .../injection_points/injection_points.c       | 16 ++++++++
 .../modules/injection_points/t/001_wait.pl    | 40 +++++++++++++++++++
 5 files changed, 78 insertions(+)
 create mode 100644 src/test/modules/injection_points/t/001_wait.pl

diff --git a/src/backend/utils/misc/injection_point.c b/src/backend/utils/misc/injection_point.c
index 0cf4d51cac..398ef2cf30 100644
--- a/src/backend/utils/misc/injection_point.c
+++ b/src/backend/utils/misc/injection_point.c
@@ -252,6 +252,26 @@ InjectionPointDetach(const char *name)
 #endif
 }
 
+/*
+ * Test if injection point is attached.
+ */
+bool
+InjectionPointIsAttach(const char *name)
+{
+#ifdef USE_INJECTION_POINTS
+	bool		found;
+
+	LWLockAcquire(InjectionPointLock, LW_EXCLUSIVE);
+	hash_search(InjectionPointHash, name, HASH_FIND, &found);
+	LWLockRelease(InjectionPointLock);
+
+	return found;
+
+#else
+	elog(ERROR, "Injection points are not supported by this build");
+#endif
+}
+
 /*
  * Execute an injection point, if defined.
  *
diff --git a/src/include/utils/injection_point.h b/src/include/utils/injection_point.h
index 55524b568f..e07f6b7024 100644
--- a/src/include/utils/injection_point.h
+++ b/src/include/utils/injection_point.h
@@ -33,5 +33,6 @@ extern void InjectionPointAttach(const char *name,
 								 const char *function);
 extern void InjectionPointRun(const char *name);
 extern void InjectionPointDetach(const char *name);
+extern bool InjectionPointIsAttach(const char *name);
 
 #endif							/* INJECTION_POINT_H */
diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile
index 2cbbae4e0a..543d2ab927 100644
--- a/src/test/modules/injection_points/Makefile
+++ b/src/test/modules/injection_points/Makefile
@@ -7,6 +7,7 @@ DATA = injection_points--1.0.sql
 PGFILEDESC = "injection_points - facility for injection points"
 
 REGRESS = injection_points
+TAP_TESTS = 1
 
 ifdef USE_PGXS
 PG_CONFIG = pg_config
diff --git a/src/test/modules/injection_points/injection_points.c b/src/test/modules/injection_points/injection_points.c
index e843e6594f..fbb30b15ad 100644
--- a/src/test/modules/injection_points/injection_points.c
+++ b/src/test/modules/injection_points/injection_points.c
@@ -18,6 +18,7 @@
 #include "postgres.h"
 
 #include "fmgr.h"
+#include "miscadmin.h"
 #include "storage/lwlock.h"
 #include "storage/shmem.h"
 #include "utils/builtins.h"
@@ -28,6 +29,7 @@ PG_MODULE_MAGIC;
 
 extern PGDLLEXPORT void injection_error(const char *name);
 extern PGDLLEXPORT void injection_notice(const char *name);
+extern PGDLLEXPORT void injection_wait(const char *name);
 
 
 /* Set of callbacks available to be attached to an injection point. */
@@ -43,6 +45,18 @@ injection_notice(const char *name)
 	elog(NOTICE, "notice triggered for injection point %s", name);
 }
 
+void
+injection_wait(const char *name)
+{
+	elog(NOTICE, "waiting triggered for injection point %s", name);
+	do
+	{
+		CHECK_FOR_INTERRUPTS();
+		pg_usleep(1000L);
+	} while (InjectionPointIsAttach(name));
+	elog(NOTICE, "waiting done for injection point %s", name);
+}
+
 /*
  * SQL function for creating an injection point.
  */
@@ -58,6 +72,8 @@ injection_points_attach(PG_FUNCTION_ARGS)
 		function = "injection_error";
 	else if (strcmp(action, "notice") == 0)
 		function = "injection_notice";
+	else if (strcmp(action, "wait") == 0)
+		function = "injection_wait";
 	else
 		elog(ERROR, "incorrect action \"%s\" for injection point creation", action);
 
diff --git a/src/test/modules/injection_points/t/001_wait.pl b/src/test/modules/injection_points/t/001_wait.pl
new file mode 100644
index 0000000000..98f7a8bcef
--- /dev/null
+++ b/src/test/modules/injection_points/t/001_wait.pl
@@ -0,0 +1,40 @@
+
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+
+use Test::More;
+
+my ($node, $result);
+
+$node = PostgreSQL::Test::Cluster->new('injection_points');
+$node->init;
+$node->start;
+$node->safe_psql('postgres', q(CREATE EXTENSION injection_points));
+
+$result = $node->psql('postgres', q(select injection_points_attach('FIRST','wait')));
+is($result, '0', 'wait injection point set');
+
+my $bg = $node->background_psql('postgres');
+
+$bg->query_until(
+	qr/start/, q(
+\echo start
+select injection_points_run('FIRST');
+select injection_points_attach('SECOND','wait');
+));
+
+$result = $node->psql('postgres', q(
+select injection_points_run('SECOND');
+select injection_points_detach('FIRST');
+));
+is($result, '0', 'wait injection point set');
+
+$bg->quit;
+
+$node->stop;
+done_testing();
-- 
2.37.1 (Apple Git-137.1)



  [application/octet-stream] v2-0001-Add-conditional-variable-to-wait-for-next-MultXac.patch (3.3K, ../../[email protected]/4-v2-0001-Add-conditional-variable-to-wait-for-next-MultXac.patch)
  download | inline diff:
From 975eb3448acbec97c14d48f21261e44aca7b3acc Mon Sep 17 00:00:00 2001
From: Andrey Borodin <[email protected]>
Date: Fri, 22 May 2020 14:13:33 +0500
Subject: [PATCH v2 1/3] Add conditional variable to wait for next MultXact
 offset in edge case

---
 src/backend/access/transam/multixact.c        | 23 ++++++++++++++++++-
 .../utils/activity/wait_event_names.txt       |  1 +
 2 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 59523be901..03fcd25d4c 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -82,6 +82,7 @@
 #include "lib/ilist.h"
 #include "miscadmin.h"
 #include "pg_trace.h"
+#include "pgstat.h"
 #include "postmaster/autovacuum.h"
 #include "storage/lmgr.h"
 #include "storage/pmsignal.h"
@@ -233,6 +234,7 @@ typedef struct MultiXactStateData
 	/* support for members anti-wraparound measures */
 	MultiXactOffset offsetStopLimit;	/* known if oldestOffsetKnown */
 
+	ConditionVariable nextoff_cv;
 	/*
 	 * Per-backend data starts here.  We have two arrays stored in the area
 	 * immediately following the MultiXactStateData struct. Each is indexed by
@@ -894,6 +896,14 @@ RecordNewMultiXact(MultiXactId multi, MultiXactOffset offset,
 	/* Exchange our lock */
 	LWLockRelease(MultiXactOffsetSLRULock);
 
+	/*
+	 *  Let everybody know the offset of this mxid is recorded now. The waiters
+	 *  are waiting for the offset of the mxid next of the target to know the
+	 *  number of members of the target mxid, so we don't need to wait for
+	 *  members of this mxid are recorded.
+	 */
+	ConditionVariableBroadcast(&MultiXactState->nextoff_cv);
+
 	LWLockAcquire(MultiXactMemberSLRULock, LW_EXCLUSIVE);
 
 	prev_pageno = -1;
@@ -1388,9 +1398,19 @@ retry:
 		if (nextMXOffset == 0)
 		{
 			/* Corner case 2: next multixact is still being filled in */
+
+			/*
+			 * The recorder of the next mxid is just before writing the offset.
+			 * Wait for the offset to be written.
+			 */
+			ConditionVariablePrepareToSleep(&MultiXactState->nextoff_cv);
+
 			LWLockRelease(MultiXactOffsetSLRULock);
 			CHECK_FOR_INTERRUPTS();
-			pg_usleep(1000L);
+
+			ConditionVariableSleep(&MultiXactState->nextoff_cv,
+								   WAIT_EVENT_NEXT_MXMEMBERS);
+			ConditionVariableCancelSleep();
 			goto retry;
 		}
 
@@ -1875,6 +1895,7 @@ MultiXactShmemInit(void)
 
 		/* Make sure we zero out the per-backend state */
 		MemSet(MultiXactState, 0, SHARED_MULTIXACT_STATE_SIZE);
+		ConditionVariableInit(&MultiXactState->nextoff_cv);
 	}
 	else
 		Assert(found);
diff --git a/src/backend/utils/activity/wait_event_names.txt b/src/backend/utils/activity/wait_event_names.txt
index a5df835dd4..1bb78d5aad 100644
--- a/src/backend/utils/activity/wait_event_names.txt
+++ b/src/backend/utils/activity/wait_event_names.txt
@@ -144,6 +144,7 @@ SYNC_REP	"Waiting for confirmation from a remote server during synchronous repli
 WAL_RECEIVER_EXIT	"Waiting for the WAL receiver to exit."
 WAL_RECEIVER_WAIT_START	"Waiting for startup process to send initial data for streaming replication."
 WAL_SUMMARY_READY	"Waiting for a new WAL summary to be generated."
+NEXT_MXMEMBERS	"Waiting for a next multixact member to be filled."
 XACT_GROUP_UPDATE	"Waiting for the group leader to update transaction status at end of a parallel operation."
 
 
-- 
2.37.1 (Apple Git-137.1)



^ permalink  raw  reply  [nested|flat] 24+ messages in thread

end of thread, other threads:[~2024-02-03 17:32 UTC | newest]

Thread overview: 24+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-12-05 14:52 [PATCH v8 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v10 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2020-12-05 14:52 [PATCH v9 3/4] COPY: flush multi-insert buffer based on accumulated size of tuples.. Justin Pryzby <[email protected]>
2024-02-03 17:32 Re: MultiXact\SLRU buffers configuration Andrey M. Borodin <[email protected]>

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox