pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix bulk table extension when copying into multiple partitions
2+ messages / 1 participants
[nested] [flat]

* pgsql: Fix bulk table extension when copying into multiple partitions
@ 2023-10-14 02:29 Andres Freund <andres@anarazel.de>
  0 siblings, 0 replies; 2+ messages in thread

From: Andres Freund @ 2023-10-14 02:29 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix bulk table extension when copying into multiple partitions

When COPYing into a partitioned table that does now permit the use of
table_multi_insert(), we could error out with
  ERROR: could not read block NN in file "base/...": read only 0 of 8192 bytes

because BulkInsertState->next_free was not reset between partitions. This
problem occurred only when not able to use table_multi_insert(), as a
dedicated BulkInsertState for each partition is used in that case.

The bug was introduced in 00d1e02be24, but it was hard to hit at that point,
as commonly bulk relation extension is not used when not using
table_multi_insert(). It became more likely after 82a4edabd27, which expanded
the use of bulk extension.

To fix the bug, reset the bulk relation extension state in BulkInsertState in
ReleaseBulkInsertStatePin(). That was added (in b1ecb9b3fcf) to tackle a very
similar issue.  Obviously the name is not quite correct, but there might be
external callers, and bulk insert state needs to be reset in precisely in the
situations that ReleaseBulkInsertStatePin() already needed to be called.

Medium term the better fix likely is to disallow reusing BulkInsertState
across relations.

Add a test that, without the fix, reproduces #18130 in most
configurations. The test also catches the problem fixed in b1ecb9b3fcf when
run with small shared_buffers.

Reported-by: Ivan Kolombet <enderstd@gmail.com>
Analyzed-by: Tom Lane <tgl@sss.pgh.pa.us>
Analyzed-by: Andres Freund <andres@anarazel.de>
Bug: #18130
Discussion: https://postgr.es/m/18130-7a86a7356a75209d%40postgresql.org
Discussion: https://postgr.es/m/257696.1695670946%40sss.pgh.pa.us
Backpatch: 16-

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/22655aa23132a0645fdcdce4b233a1fff0c0cf8f

Modified Files
--------------
src/backend/access/heap/heapam.c   | 11 +++++++++++
src/test/regress/expected/copy.out | 37 +++++++++++++++++++++++++++++++++++++
src/test/regress/sql/copy.sql      | 37 +++++++++++++++++++++++++++++++++++++
3 files changed, 85 insertions(+)



^ permalink  raw  reply  [nested|flat] 2+ messages in thread

* pgsql: Fix bulk table extension when copying into multiple partitions
@ 2023-10-14 02:29 Andres Freund <andres@anarazel.de>
  0 siblings, 0 replies; 2+ messages in thread

From: Andres Freund @ 2023-10-14 02:29 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix bulk table extension when copying into multiple partitions

When COPYing into a partitioned table that does now permit the use of
table_multi_insert(), we could error out with
  ERROR: could not read block NN in file "base/...": read only 0 of 8192 bytes

because BulkInsertState->next_free was not reset between partitions. This
problem occurred only when not able to use table_multi_insert(), as a
dedicated BulkInsertState for each partition is used in that case.

The bug was introduced in 00d1e02be24, but it was hard to hit at that point,
as commonly bulk relation extension is not used when not using
table_multi_insert(). It became more likely after 82a4edabd27, which expanded
the use of bulk extension.

To fix the bug, reset the bulk relation extension state in BulkInsertState in
ReleaseBulkInsertStatePin(). That was added (in b1ecb9b3fcf) to tackle a very
similar issue.  Obviously the name is not quite correct, but there might be
external callers, and bulk insert state needs to be reset in precisely in the
situations that ReleaseBulkInsertStatePin() already needed to be called.

Medium term the better fix likely is to disallow reusing BulkInsertState
across relations.

Add a test that, without the fix, reproduces #18130 in most
configurations. The test also catches the problem fixed in b1ecb9b3fcf when
run with small shared_buffers.

Reported-by: Ivan Kolombet <enderstd@gmail.com>
Analyzed-by: Tom Lane <tgl@sss.pgh.pa.us>
Analyzed-by: Andres Freund <andres@anarazel.de>
Bug: #18130
Discussion: https://postgr.es/m/18130-7a86a7356a75209d%40postgresql.org
Discussion: https://postgr.es/m/257696.1695670946%40sss.pgh.pa.us
Backpatch: 16-

Branch
------
REL_16_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/0002feb8209618e5a9e23e03fe4aa31bc4006f01

Modified Files
--------------
src/backend/access/heap/heapam.c   | 11 +++++++++++
src/test/regress/expected/copy.out | 37 +++++++++++++++++++++++++++++++++++++
src/test/regress/sql/copy.sql      | 37 +++++++++++++++++++++++++++++++++++++
3 files changed, 85 insertions(+)



^ permalink  raw  reply  [nested|flat] 2+ messages in thread


end of thread, other threads:[~2023-10-14 02:29 UTC | newest]

Thread overview: 2+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-10-14 02:29 pgsql: Fix bulk table extension when copying into multiple partitions Andres Freund <andres@anarazel.de>
2023-10-14 02:29 pgsql: Fix bulk table extension when copying into multiple partitions Andres Freund <andres@anarazel.de>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox