pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL 7+ messages / 1 participants [nested] [flat]
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/97ddda8a82ac470ae581d0eb485b6577707678bc Modified Files -------------- src/backend/commands/tablespace.c | 42 +++++++++++++++------------------ src/test/recovery/t/018_wal_optimize.pl | 31 ++++++++++++++++++++---- 2 files changed, 46 insertions(+), 27 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/5513c09c89993cb0b2c1a4222b15722bcf71844b Modified Files -------------- src/backend/commands/tablespace.c | 42 +++++++++++++++------------------ src/test/recovery/t/018_wal_optimize.pl | 31 ++++++++++++++++++++---- 2 files changed, 46 insertions(+), 27 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/b18669f5e652f1d0c1dbf332fd7cd24a38c7ed31 Modified Files -------------- src/backend/commands/tablespace.c | 42 +++++++++++++++------------------ src/test/recovery/t/018_wal_optimize.pl | 31 ++++++++++++++++++++---- 2 files changed, 46 insertions(+), 27 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a494f10232645c3456bee7f6fbea5546f8a8166d Modified Files -------------- src/backend/commands/tablespace.c | 42 ++++++++++++++++++--------------------- 1 file changed, 19 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/6ebd2426bda2260fc9bc33555da81dfc8c623f32 Modified Files -------------- src/backend/commands/tablespace.c | 42 ++++++++++++++++++--------------------- 1 file changed, 19 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/f11c1bb17099834a3640a750b263e08f5e6f8135 Modified Files -------------- src/backend/commands/tablespace.c | 42 ++++++++++++++++++--------------------- 1 file changed, 19 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
* pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL @ 2021-08-28 06:51 Noah Misch <noah@leadboat.com> 0 siblings, 0 replies; 7+ messages in thread From: Noah Misch @ 2021-08-28 06:51 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Fix data loss in wal_level=minimal crash recovery of CREATE TABLESPACE. If the system crashed between CREATE TABLESPACE and the next checkpoint, the result could be some files in the tablespace unexpectedly containing no rows. Affected files would be those for which the system did not write WAL; see the wal_skip_threshold documentation. Before v13, a different set of conditions governed the writing of WAL; see v12's <sect2 id="populate-pitr">. (The v12 conditions were broader in some ways and narrower in others.) Users may want to audit non-default tablespaces for unexpected short files. The bug could have truncated an index without affecting the associated table, and reindexing the index would fix that particular problem. This fixes the bug by making create_tablespace_directories() more like TablespaceCreateDbspace(). create_tablespace_directories() was recursively removing tablespace contents, reasoning that WAL redo would recreate everything removed that way. That assumption holds for other wal_level values. Under wal_level=minimal, the old approach could delete files for which no other copy existed. Back-patch to 9.6 (all supported versions). Reviewed by Robert Haas and Prabhat Sahu. Reported by Robert Haas. Discussion: https://postgr.es/m/CA+TgmoaLO9ncuwvr2nN-J4VEP5XyAcy=zKiHxQzBbFRxxGxm0w@mail.gmail.com Branch ------ REL9_6_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/978998dbde2aac0e95680a35afcc1911cf1fa93f Modified Files -------------- src/backend/commands/tablespace.c | 42 ++++++++++++++++++--------------------- 1 file changed, 19 insertions(+), 23 deletions(-) ^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2021-08-28 06:51 UTC | newest] Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com> 2021-08-28 06:51 pgsql: Fix data loss in wal_level=minimal crash recovery of CREATE TABL Noah Misch <noah@leadboat.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox