pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Avoid deleting critical WAL segments during pg_rewind 5+ messages / 1 participants [nested] [flat]
* pgsql: Avoid deleting critical WAL segments during pg_rewind @ 2024-11-15 12:10 Álvaro Herrera <alvherre@alvh.no-ip.org> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2024-11-15 12:10 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Avoid deleting critical WAL segments during pg_rewind Previously, in unlucky cases, it was possible for pg_rewind to remove certain WAL segments from the rewound demoted primary. In particular this happens if those files have been marked for archival (i.e., their .ready files were created) but not yet archived; the newly promoted node no longer has such files because of them having been recycled, but they are likely critical for recovery in the demoted node. If pg_rewind removes them, recovery is not possible anymore. Fix this by maintaining a hash table of files in this situation in the scan that looks for a checkpoint, which the decide_file_actions phase can consult so that it knows to preserve them. Backpatch to 14. The problem also exists in 13, but that branch was not blessed with commit eb00f1d4bf96, so this patch is difficult to apply there. Users of older releases will just have to continue to be extra careful when rewinding. Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com> Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/90bcc7c2db1d346e1153748e627c2f922b67eae7 Modified Files -------------- src/bin/pg_rewind/filemap.c | 84 ++++++++++++++++++++++++--- src/bin/pg_rewind/filemap.h | 3 + src/bin/pg_rewind/meson.build | 1 + src/bin/pg_rewind/parsexlog.c | 21 +++++++ src/bin/pg_rewind/pg_rewind.c | 3 + src/bin/pg_rewind/t/010_keep_recycled_wals.pl | 62 ++++++++++++++++++++ src/tools/pgindent/typedefs.list | 2 + 7 files changed, 169 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Avoid deleting critical WAL segments during pg_rewind @ 2024-11-15 12:10 Álvaro Herrera <alvherre@alvh.no-ip.org> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2024-11-15 12:10 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Avoid deleting critical WAL segments during pg_rewind Previously, in unlucky cases, it was possible for pg_rewind to remove certain WAL segments from the rewound demoted primary. In particular this happens if those files have been marked for archival (i.e., their .ready files were created) but not yet archived; the newly promoted node no longer has such files because of them having been recycled, but they are likely critical for recovery in the demoted node. If pg_rewind removes them, recovery is not possible anymore. Fix this by maintaining a hash table of files in this situation in the scan that looks for a checkpoint, which the decide_file_actions phase can consult so that it knows to preserve them. Backpatch to 14. The problem also exists in 13, but that branch was not blessed with commit eb00f1d4bf96, so this patch is difficult to apply there. Users of older releases will just have to continue to be extra careful when rewinding. Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com> Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com Branch ------ REL_17_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/cb844d66b5872782481deebc793919c7d531c032 Modified Files -------------- src/bin/pg_rewind/filemap.c | 84 ++++++++++++++++++++++++--- src/bin/pg_rewind/filemap.h | 3 + src/bin/pg_rewind/meson.build | 1 + src/bin/pg_rewind/parsexlog.c | 21 +++++++ src/bin/pg_rewind/pg_rewind.c | 3 + src/bin/pg_rewind/t/010_keep_recycled_wals.pl | 62 ++++++++++++++++++++ src/tools/pgindent/typedefs.list | 2 + 7 files changed, 169 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Avoid deleting critical WAL segments during pg_rewind @ 2024-11-15 12:10 Álvaro Herrera <alvherre@alvh.no-ip.org> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2024-11-15 12:10 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Avoid deleting critical WAL segments during pg_rewind Previously, in unlucky cases, it was possible for pg_rewind to remove certain WAL segments from the rewound demoted primary. In particular this happens if those files have been marked for archival (i.e., their .ready files were created) but not yet archived; the newly promoted node no longer has such files because of them having been recycled, but they are likely critical for recovery in the demoted node. If pg_rewind removes them, recovery is not possible anymore. Fix this by maintaining a hash table of files in this situation in the scan that looks for a checkpoint, which the decide_file_actions phase can consult so that it knows to preserve them. Backpatch to 14. The problem also exists in 13, but that branch was not blessed with commit eb00f1d4bf96, so this patch is difficult to apply there. Users of older releases will just have to continue to be extra careful when rewinding. Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com> Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com Branch ------ REL_16_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ea1649c352f67bde3eafc0005fb70e3766f1153a Modified Files -------------- src/bin/pg_rewind/filemap.c | 84 ++++++++++++++++++++++++--- src/bin/pg_rewind/filemap.h | 3 + src/bin/pg_rewind/meson.build | 1 + src/bin/pg_rewind/parsexlog.c | 21 +++++++ src/bin/pg_rewind/pg_rewind.c | 3 + src/bin/pg_rewind/t/010_keep_recycled_wals.pl | 62 ++++++++++++++++++++ src/tools/pgindent/typedefs.list | 2 + 7 files changed, 169 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Avoid deleting critical WAL segments during pg_rewind @ 2024-11-15 12:10 Álvaro Herrera <alvherre@alvh.no-ip.org> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2024-11-15 12:10 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Avoid deleting critical WAL segments during pg_rewind Previously, in unlucky cases, it was possible for pg_rewind to remove certain WAL segments from the rewound demoted primary. In particular this happens if those files have been marked for archival (i.e., their .ready files were created) but not yet archived; the newly promoted node no longer has such files because of them having been recycled, but they are likely critical for recovery in the demoted node. If pg_rewind removes them, recovery is not possible anymore. Fix this by maintaining a hash table of files in this situation in the scan that looks for a checkpoint, which the decide_file_actions phase can consult so that it knows to preserve them. Backpatch to 14. The problem also exists in 13, but that branch was not blessed with commit eb00f1d4bf96, so this patch is difficult to apply there. Users of older releases will just have to continue to be extra careful when rewinding. Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com> Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com Branch ------ REL_15_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/e28cf2fbc222a607377813590e4bee448fcf0a29 Modified Files -------------- src/bin/pg_rewind/filemap.c | 84 ++++++++++++++++++++++++--- src/bin/pg_rewind/filemap.h | 3 + src/bin/pg_rewind/parsexlog.c | 21 +++++++ src/bin/pg_rewind/pg_rewind.c | 3 + src/bin/pg_rewind/t/010_keep_recycled_wals.pl | 62 ++++++++++++++++++++ src/tools/pgindent/typedefs.list | 2 + 6 files changed, 168 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
* pgsql: Avoid deleting critical WAL segments during pg_rewind @ 2024-11-15 12:10 Álvaro Herrera <alvherre@alvh.no-ip.org> 0 siblings, 0 replies; 5+ messages in thread From: Álvaro Herrera @ 2024-11-15 12:10 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Avoid deleting critical WAL segments during pg_rewind Previously, in unlucky cases, it was possible for pg_rewind to remove certain WAL segments from the rewound demoted primary. In particular this happens if those files have been marked for archival (i.e., their .ready files were created) but not yet archived; the newly promoted node no longer has such files because of them having been recycled, but they are likely critical for recovery in the demoted node. If pg_rewind removes them, recovery is not possible anymore. Fix this by maintaining a hash table of files in this situation in the scan that looks for a checkpoint, which the decide_file_actions phase can consult so that it knows to preserve them. Backpatch to 14. The problem also exists in 13, but that branch was not blessed with commit eb00f1d4bf96, so this patch is difficult to apply there. Users of older releases will just have to continue to be extra careful when rewinding. Co-authored-by: Полина Бунгина (Polina Bungina) <bungina@gmail.com> Co-authored-by: Alexander Kukushkin <cyberdemn@gmail.com> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Reviewed-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com> Discussion: https://postgr.es/m/CAAtGL4AhzmBRsEsaDdz7065T+k+BscNadfTqP1NcPmsqwA5HBw@mail.gmail.com Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/ba25358413e43b7ea442d1c80334d73940be060b Modified Files -------------- src/bin/pg_rewind/filemap.c | 84 ++++++++++++++++++++++++--- src/bin/pg_rewind/filemap.h | 3 + src/bin/pg_rewind/parsexlog.c | 21 +++++++ src/bin/pg_rewind/pg_rewind.c | 3 + src/bin/pg_rewind/t/010_keep_recycled_wals.pl | 62 ++++++++++++++++++++ src/tools/pgindent/typedefs.list | 2 + 6 files changed, 168 insertions(+), 7 deletions(-) ^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2024-11-15 12:10 UTC | newest] Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-11-15 12:10 pgsql: Avoid deleting critical WAL segments during pg_rewind Álvaro Herrera <alvherre@alvh.no-ip.org> 2024-11-15 12:10 pgsql: Avoid deleting critical WAL segments during pg_rewind Álvaro Herrera <alvherre@alvh.no-ip.org> 2024-11-15 12:10 pgsql: Avoid deleting critical WAL segments during pg_rewind Álvaro Herrera <alvherre@alvh.no-ip.org> 2024-11-15 12:10 pgsql: Avoid deleting critical WAL segments during pg_rewind Álvaro Herrera <alvherre@alvh.no-ip.org> 2024-11-15 12:10 pgsql: Avoid deleting critical WAL segments during pg_rewind Álvaro Herrera <alvherre@alvh.no-ip.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox