public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v3 3/3] Fix sizing of tape read buffers.
26+ messages / 3 participants
[nested] [flat]
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* [PATCH v3 3/3] Fix sizing of tape read buffers.
@ 2021-01-22 20:59 Heikki Linnakangas <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Heikki Linnakangas @ 2021-01-22 20:59 UTC (permalink / raw)
---
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/backend/utils/sort/tuplesort.c b/src/backend/utils/sort/tuplesort.c
index 14104643e78..aff2f201f52 100644
--- a/src/backend/utils/sort/tuplesort.c
+++ b/src/backend/utils/sort/tuplesort.c
@@ -2658,7 +2658,7 @@ merge_read_buffer_size(int64 avail_mem, int nInputTapes, int nInputRuns,
* we derive the input buffer size from the amount of memory available,
* and M and N.
*/
- return (avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes;
+ return Max((avail_mem - TAPE_BUFFER_OVERHEAD * nOutputTapes) / nInputTapes, 0);
}
/*
--
2.29.2
--------------631E0A48DCAEA386819DFCEC--
^ permalink raw reply [nested|flat] 26+ messages in thread
* buffer refcount leak in foreign batch insert code
@ 2023-04-21 10:07 Alexander Pyhalov <[email protected]>
2023-04-21 10:16 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
0 siblings, 1 reply; 26+ messages in thread
From: Alexander Pyhalov @ 2023-04-21 10:07 UTC (permalink / raw)
To: pgsql-hackers
Hi.
We've found that in cases like the one attached, when we insert into
foreign partition with batch_size set, buffer refcount leak is detected.
The above example we see a dozen of similar messages:
repro_small.sql:31: WARNING: buffer refcount leak: [14621]
(rel=base/16718/16732, blockNum=54, flags=0x93800000
The issue was introduced in the following commit
commit b676ac443b6a83558d4701b2dd9491c0b37e17c4
Author: Tomas Vondra <[email protected]>
Date: Fri Jun 11 20:19:48 2021 +0200
Optimize creation of slots for FDW bulk inserts
In this commit we avoid recreating slots for each batch. But it seems
that created slots should still be cleared in the end of
ExecBatchInsert().
At least the attached patch seems to fix the issue.
--
Best regards,
Alexander Pyhalov,
Postgres Professional
CREATE EXTENSION postgres_fdw;
DO $d$
BEGIN
EXECUTE $$CREATE SERVER loopback FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (dbname '$$||current_database()||$$',
port '$$||current_setting('port')||$$'
)$$;
END
$d$;
CREATE USER MAPPING FOR CURRENT_USER SERVER loopback;
CREATE TABLE local_data (id int, data text);
INSERT INTO local_data SELECT i, 'test'|| i FROM generate_series(1, 10000) i;
CREATE TABLE sharded_data (id int, data text) PARTITION BY HASH(id);
CREATE TABLE sharded_data_p0 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 0);
CREATE TABLE sharded_data_p1_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p1 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 1)
SERVER loopback OPTIONS (table_name 'sharded_data_p1_remote');
CREATE TABLE sharded_data_p2_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p2 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 2)
SERVER loopback OPTIONS (table_name 'sharded_data_p2_remote');
CREATE TABLE sharded_data_p3_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p3 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 3)
SERVER loopback OPTIONS (table_name 'sharded_data_p3_remote');
insert into sharded_data select * from local_data ;
delete from sharded_data;
alter server loopback options (add batch_size '100');
insert into sharded_data select * from local_data ;
Attachments:
[text/plain] repro_small.sql (1.4K, ../../[email protected]/2-repro_small.sql)
download | inline:
CREATE EXTENSION postgres_fdw;
DO $d$
BEGIN
EXECUTE $$CREATE SERVER loopback FOREIGN DATA WRAPPER postgres_fdw
OPTIONS (dbname '$$||current_database()||$$',
port '$$||current_setting('port')||$$'
)$$;
END
$d$;
CREATE USER MAPPING FOR CURRENT_USER SERVER loopback;
CREATE TABLE local_data (id int, data text);
INSERT INTO local_data SELECT i, 'test'|| i FROM generate_series(1, 10000) i;
CREATE TABLE sharded_data (id int, data text) PARTITION BY HASH(id);
CREATE TABLE sharded_data_p0 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 0);
CREATE TABLE sharded_data_p1_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p1 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 1)
SERVER loopback OPTIONS (table_name 'sharded_data_p1_remote');
CREATE TABLE sharded_data_p2_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p2 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 2)
SERVER loopback OPTIONS (table_name 'sharded_data_p2_remote');
CREATE TABLE sharded_data_p3_remote (id int, data text);
CREATE FOREIGN TABLE sharded_data_p3 PARTITION OF sharded_data FOR VALUES WITH (modulus 4, remainder 3)
SERVER loopback OPTIONS (table_name 'sharded_data_p3_remote');
insert into sharded_data select * from local_data ;
delete from sharded_data;
alter server loopback options (add batch_size '100');
insert into sharded_data select * from local_data ;
[text/x-diff] 0001-Fix-buffer-refcount-leak.patch (867B, ../../[email protected]/3-0001-Fix-buffer-refcount-leak.patch)
download | inline diff:
From 12d1fbf56c1c82d4659da484c2207539f396aac9 Mon Sep 17 00:00:00 2001
From: Alexander Pyhalov <[email protected]>
Date: Fri, 21 Apr 2023 12:54:41 +0300
Subject: [PATCH] Fix buffer refcount leak
The buffer refcount leak was introduced
in b676ac443b6a83558d4701b2dd9491c0b37e17c4
---
src/backend/executor/nodeModifyTable.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 6aa8c03defb..14ab8382183 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1261,6 +1261,12 @@ ExecBatchInsert(ModifyTableState *mtstate,
if (canSetTag && numInserted > 0)
estate->es_processed += numInserted;
+
+ for (i = 0; i < numSlots; i++)
+ {
+ ExecClearTuple(slots[i]);
+ ExecClearTuple(planSlots[i]);
+ }
}
/*
--
2.34.1
^ permalink raw reply [nested|flat] 26+ messages in thread
* Re: buffer refcount leak in foreign batch insert code
2023-04-21 10:07 buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
@ 2023-04-21 10:16 ` Michael Paquier <[email protected]>
2023-04-24 00:57 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
0 siblings, 1 reply; 26+ messages in thread
From: Michael Paquier @ 2023-04-21 10:16 UTC (permalink / raw)
To: Alexander Pyhalov <[email protected]>; +Cc: pgsql-hackers
On Fri, Apr 21, 2023 at 01:07:03PM +0300, Alexander Pyhalov wrote:
> We've found that in cases like the one attached, when we insert into foreign
> partition with batch_size set, buffer refcount leak is detected.
>
> The above example we see a dozen of similar messages:
>
> repro_small.sql:31: WARNING: buffer refcount leak: [14621]
> (rel=base/16718/16732, blockNum=54, flags=0x93800000
Indeed, nice repro! That's obviously wrong, I'll look into that.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 26+ messages in thread
* Re: buffer refcount leak in foreign batch insert code
2023-04-21 10:07 buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
2023-04-21 10:16 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
@ 2023-04-24 00:57 ` Michael Paquier <[email protected]>
2023-04-25 01:30 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
0 siblings, 1 reply; 26+ messages in thread
From: Michael Paquier @ 2023-04-24 00:57 UTC (permalink / raw)
To: Alexander Pyhalov <[email protected]>; +Cc: pgsql-hackers
On Fri, Apr 21, 2023 at 07:16:03PM +0900, Michael Paquier wrote:
> On Fri, Apr 21, 2023 at 01:07:03PM +0300, Alexander Pyhalov wrote:
>> We've found that in cases like the one attached, when we insert into foreign
>> partition with batch_size set, buffer refcount leak is detected.
>>
>> The above example we see a dozen of similar messages:
>>
>> repro_small.sql:31: WARNING: buffer refcount leak: [14621]
>> (rel=base/16718/16732, blockNum=54, flags=0x93800000
This reminds me a lot of the recent multi-insert logic added to
various DDL code paths for catalogs, bref..
The number of slots ready for a batch is tracked by ri_NumSlots, and
it is reset to 0 each time a batch has been processed. How about
resetting the counter at the same place as the slots are cleared, at
the end of ExecBatchInsert() as the same time as when the slots are
cleared?
I was wondering as well about resetting the slot just before copying
something into ri_Slots in ExecInsert() (actually close to the slot
copy), which is something that would make the operation cheaper for
large batches because the last batch could be cleaned up with
ExecEndModifyTable(), but this makes the code much messier when a
tuple is added into one of the slots as we would need to switch
back-and-forth with es_query_cxt from what I can see, because th
batches are inserted before any slot initialization is done. In
short, I'm okay with what's proposed here and clean up things at the
same time as ri_NumSlots.
Another thing was the interaction of this change with triggers
(delete, insert with returning and batches to flush pending inserts,
etc.), and that looked correct to me (I have plugged in some of these
triggers noisy on notices on the relations of the partitions tree).
Self-reminder: the tests of postgres_fdw are rather long now, perhaps
these should be split into more files in the future..
The attached is what I am finishing with, with a minimal regression
test added to postgres_fdw. Two partitions are enough. Alexander,
what do you think?
--
Michael
Attachments:
[text/x-diff] v2-0001-Fix-buffer-refcount-leak.patch (4.4K, ../../ZEXT5sTzMJe%[email protected]/2-v2-0001-Fix-buffer-refcount-leak.patch)
download | inline diff:
From 6d9da4ecfbca7addd0450c2f5bb359a47a0f4e98 Mon Sep 17 00:00:00 2001
From: Michael Paquier <[email protected]>
Date: Mon, 24 Apr 2023 09:51:48 +0900
Subject: [PATCH v2] Fix buffer refcount leak
The buffer refcount leak was introduced
in b676ac443b6a83558d4701b2dd9491c0b37e17c4.
---
src/backend/executor/nodeModifyTable.c | 10 +++++++--
.../postgres_fdw/expected/postgres_fdw.out | 22 +++++++++++++++++++
contrib/postgres_fdw/sql/postgres_fdw.sql | 18 +++++++++++++++
3 files changed, 48 insertions(+), 2 deletions(-)
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 6aa8c03def..dc1a2ec551 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -856,7 +856,6 @@ ExecInsert(ModifyTableContext *context,
resultRelInfo->ri_PlanSlots,
resultRelInfo->ri_NumSlots,
estate, canSetTag);
- resultRelInfo->ri_NumSlots = 0;
flushed = true;
}
@@ -1261,6 +1260,14 @@ ExecBatchInsert(ModifyTableState *mtstate,
if (canSetTag && numInserted > 0)
estate->es_processed += numInserted;
+
+ /* Clean up all the slots, ready for the next batch */
+ for (i = 0; i < numSlots; i++)
+ {
+ ExecClearTuple(slots[i]);
+ ExecClearTuple(planSlots[i]);
+ }
+ resultRelInfo->ri_NumSlots = 0;
}
/*
@@ -1284,7 +1291,6 @@ ExecPendingInserts(EState *estate)
resultRelInfo->ri_PlanSlots,
resultRelInfo->ri_NumSlots,
estate, mtstate->canSetTag);
- resultRelInfo->ri_NumSlots = 0;
}
list_free(estate->es_insert_pending_result_relations);
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out b/contrib/postgres_fdw/expected/postgres_fdw.out
index fd5752bd5b..826baac9f1 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -6930,6 +6930,28 @@ select * from grem1;
(2 rows)
delete from grem1;
+-- batch insert with foreign partitions.
+-- This schema uses two partitions, one local and one remote with a modulo
+-- to loop across all of them in batches.
+create table tab_batch_local (id int, data text);
+insert into tab_batch_local select i, 'test'|| i from generate_series(1, 45) i;
+create table tab_batch_sharded (id int, data text) partition by hash(id);
+create table tab_batch_sharded_p0 partition of tab_batch_sharded
+ for values with (modulus 2, remainder 0);
+create table tab_batch_sharded_p1_remote (id int, data text);
+create foreign table tab_batch_sharded_p1 partition of tab_batch_sharded
+ for values with (modulus 2, remainder 1)
+ server loopback options (table_name 'tab_batch_sharded_p1_remote');
+insert into tab_batch_sharded select * from tab_batch_local;
+select count(*) from tab_batch_sharded;
+ count
+-------
+ 45
+(1 row)
+
+drop table tab_batch_local;
+drop table tab_batch_sharded;
+drop table tab_batch_sharded_p1_remote;
alter server loopback options (drop batch_size);
-- ===================================================================
-- test local triggers
diff --git a/contrib/postgres_fdw/sql/postgres_fdw.sql b/contrib/postgres_fdw/sql/postgres_fdw.sql
index c05046f867..15f3af6c29 100644
--- a/contrib/postgres_fdw/sql/postgres_fdw.sql
+++ b/contrib/postgres_fdw/sql/postgres_fdw.sql
@@ -1657,6 +1657,24 @@ insert into grem1 (a) values (1), (2);
select * from gloc1;
select * from grem1;
delete from grem1;
+-- batch insert with foreign partitions.
+-- This schema uses two partitions, one local and one remote with a modulo
+-- to loop across all of them in batches.
+create table tab_batch_local (id int, data text);
+insert into tab_batch_local select i, 'test'|| i from generate_series(1, 45) i;
+create table tab_batch_sharded (id int, data text) partition by hash(id);
+create table tab_batch_sharded_p0 partition of tab_batch_sharded
+ for values with (modulus 2, remainder 0);
+create table tab_batch_sharded_p1_remote (id int, data text);
+create foreign table tab_batch_sharded_p1 partition of tab_batch_sharded
+ for values with (modulus 2, remainder 1)
+ server loopback options (table_name 'tab_batch_sharded_p1_remote');
+insert into tab_batch_sharded select * from tab_batch_local;
+select count(*) from tab_batch_sharded;
+drop table tab_batch_local;
+drop table tab_batch_sharded;
+drop table tab_batch_sharded_p1_remote;
+
alter server loopback options (drop batch_size);
-- ===================================================================
--
2.40.0
[application/pgp-signature] signature.asc (833B, ../../ZEXT5sTzMJe%[email protected]/3-signature.asc)
download
^ permalink raw reply [nested|flat] 26+ messages in thread
* Re: buffer refcount leak in foreign batch insert code
2023-04-21 10:07 buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
2023-04-21 10:16 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-04-24 00:57 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
@ 2023-04-25 01:30 ` Michael Paquier <[email protected]>
2023-05-02 07:03 ` Re: buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
0 siblings, 1 reply; 26+ messages in thread
From: Michael Paquier @ 2023-04-25 01:30 UTC (permalink / raw)
To: Alexander Pyhalov <[email protected]>; +Cc: pgsql-hackers
On Mon, Apr 24, 2023 at 09:57:10AM +0900, Michael Paquier wrote:
> The attached is what I am finishing with, with a minimal regression
> test added to postgres_fdw. Two partitions are enough.
Well, I have gone through that again this morning, and applied the fix
down to 14. The buildfarm is digesting it fine.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 26+ messages in thread
* Re: buffer refcount leak in foreign batch insert code
2023-04-21 10:07 buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
2023-04-21 10:16 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-04-24 00:57 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-04-25 01:30 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
@ 2023-05-02 07:03 ` Alexander Pyhalov <[email protected]>
0 siblings, 0 replies; 26+ messages in thread
From: Alexander Pyhalov @ 2023-05-02 07:03 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: pgsql-hackers
Michael Paquier писал 2023-04-25 04:30:
> On Mon, Apr 24, 2023 at 09:57:10AM +0900, Michael Paquier wrote:
>> The attached is what I am finishing with, with a minimal regression
>> test added to postgres_fdw. Two partitions are enough.
>
> Well, I have gone through that again this morning, and applied the fix
> down to 14. The buildfarm is digesting it fine.
> --
> Michael
Thank you. Sorry for the late response, was on vacation.
--
Best regards,
Alexander Pyhalov,
Postgres Professional
^ permalink raw reply [nested|flat] 26+ messages in thread
end of thread, other threads:[~2023-05-02 07:03 UTC | newest]
Thread overview: 26+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2021-01-22 20:59 [PATCH v3 3/3] Fix sizing of tape read buffers. Heikki Linnakangas <[email protected]>
2023-04-21 10:07 buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
2023-04-21 10:16 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-04-24 00:57 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-04-25 01:30 ` Re: buffer refcount leak in foreign batch insert code Michael Paquier <[email protected]>
2023-05-02 07:03 ` Re: buffer refcount leak in foreign batch insert code Alexander Pyhalov <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox