pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Clean up error handling in pg_basebackup's walmethods.c.
6+ messages / 1 participants
[nested] [flat]

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/248c3a937dd018a72095f407cff727c9f08db0c1

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 235 ++++++++++++++++++++++++++-----------
1 file changed, 169 insertions(+), 66 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/53c4a580e4ab90295a0a3f0814089ac978901b3f

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 200 ++++++++++++++++++++++++++-----------
1 file changed, 141 insertions(+), 59 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/c8b5221b57677231511039e943150a74353ed44d

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 204 ++++++++++++++++++++++++++-----------
1 file changed, 144 insertions(+), 60 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/8378dad4c579438f40c42b8514f2837c38094ec9

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 200 ++++++++++++++++++++++++++-----------
1 file changed, 139 insertions(+), 61 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a414eb850dd8cf9337dd5e8000e2f4e68b61769f

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 198 ++++++++++++++++++++++++++-----------
1 file changed, 138 insertions(+), 60 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread

* pgsql: Clean up error handling in pg_basebackup's walmethods.c.
@ 2021-11-17 19:16 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 6+ messages in thread

From: Tom Lane @ 2021-11-17 19:16 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Clean up error handling in pg_basebackup's walmethods.c.

The error handling here was a mess, as a result of a fundamentally
bad design (relying on errno to keep its value much longer than is
safe to assume) as well as a lot of just plain sloppiness, both as
to noticing errors at all and as to reporting the correct errno.
Moreover, the recent addition of LZ4 compression broke things
completely, because liblz4 doesn't use errno to report errors.

To improve matters, keep the error state in the DirectoryMethodData or
TarMethodData struct, and add a string field so we can handle cases
that don't set errno.  (The tar methods already had a version of this,
but it can be done more efficiently since all these cases use a
constant error string.)  Make the dir and tar methods handle errors
in basically identical ways, which they didn't before.

This requires copying errno into the state struct in a lot of places,
which is a bit tedious, but it has the virtue that we can get rid of
ad-hoc code to save and restore errno in a number of places ... not
to mention that it fixes other places that should've saved/restored
errno but neglected to.

In passing, fix some pointlessly static buffers to be ordinary
local variables.

There remains an issue about exactly how to handle errors from
fsync(), but that seems like material for its own patch.

While the LZ4 problems are new, all the rest of this is fixes for
old bugs, so backpatch to v10 where walmethods.c was introduced.

Patch by me; thanks to Michael Paquier for review.

Discussion: https://postgr.es/m/1343113.1636489231@sss.pgh.pa.us

Branch
------
REL_10_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/13799bb93866a3fbe02c0811b9c7e3570700a7b5

Modified Files
--------------
src/bin/pg_basebackup/walmethods.c | 198 ++++++++++++++++++++++++++-----------
1 file changed, 138 insertions(+), 60 deletions(-)



^ permalink  raw  reply  [nested|flat] 6+ messages in thread


end of thread, other threads:[~2021-11-17 19:16 UTC | newest]

Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>
2021-11-17 19:16 pgsql: Clean up error handling in pg_basebackup's walmethods.c. Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox