pg.ddx.io pgsql-committers@postgresql.org mailing list archivehelp / color / mirror / Atom feed
pgsql: Clean up error handling in pg_basebackup's walmethods.c. 6+ messages / 1 participants [nested] [flat]
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ REL_14_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/53c4a580e4ab90295a0a3f0814089ac978901b3f Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 200 ++++++++++++++++++++++++++----------- 1 file changed, 141 insertions(+), 59 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ REL_10_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/13799bb93866a3fbe02c0811b9c7e3570700a7b5 Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 198 ++++++++++++++++++++++++++----------- 1 file changed, 138 insertions(+), 60 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/a414eb850dd8cf9337dd5e8000e2f4e68b61769f Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 198 ++++++++++++++++++++++++++----------- 1 file changed, 138 insertions(+), 60 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/248c3a937dd018a72095f407cff727c9f08db0c1 Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 235 ++++++++++++++++++++++++++----------- 1 file changed, 169 insertions(+), 66 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ REL_12_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/8378dad4c579438f40c42b8514f2837c38094ec9 Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 200 ++++++++++++++++++++++++++----------- 1 file changed, 139 insertions(+), 61 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
* pgsql: Clean up error handling in pg_basebackup's walmethods.c. @ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us> 0 siblings, 0 replies; 6+ messages in thread From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw) To: pgsql-committers@lists.postgresql.org Clean up error handling in pg_basebackup's walmethods.c. The error handling here was a mess, as a result of a fundamentally bad design (relying on errno to keep its value much longer than is safe to assume) as well as a lot of just plain sloppiness, both as to noticing errors at all and as to reporting the correct errno. Moreover, the recent addition of LZ4 compression broke things completely, because liblz4 doesn't use errno to report errors. To improve matters, keep the error state in the DirectoryMethodData or TarMethodData struct, and add a string field so we can handle cases that don't set errno. (The tar methods already had a version of this, but it can be done more efficiently since all these cases use a constant error string.) Make the dir and tar methods handle errors in basically identical ways, which they didn't before. This requires copying errno into the state struct in a lot of places, which is a bit tedious, but it has the virtue that we can get rid of ad-hoc code to save and restore errno in a number of places ... not to mention that it fixes other places that should've saved/restored errno but neglected to. In passing, fix some pointlessly static buffers to be ordinary local variables. There remains an issue about exactly how to handle errors from fsync(), but that seems like material for its own patch. While the LZ4 problems are new, all the rest of this is fixes for old bugs, so backpatch to v10 where walmethods.c was introduced. Patch by me; thanks to Michael Paquier for review. Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us Branch ------ REL_13_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/c8b5221b57677231511039e943150a74353ed44d Modified Files -------------- src/bin/pg_basebackup/walmethods.c | 204 ++++++++++++++++++++++++++----------- 1 file changed, 144 insertions(+), 60 deletions(-) ^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2021-11-17 19:16 UTC | newest] Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us> 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us> 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us> 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us> 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us> 2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions for how to clone and mirror all data and code used for this inbox