pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #16866: pg_basebackup Windows Server 2016
5+ messages / 5 participants
[nested] [flat]
* BUG #16866: pg_basebackup Windows Server 2016
@ 2021-02-15 15:12 PG Bug reporting form <noreply@postgresql.org>
2021-02-16 01:00 ` Re: BUG #16866: pg_basebackup Windows Server 2016 Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 5+ messages in thread
From: PG Bug reporting form @ 2021-02-15 15:12 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: ivan.salvato@gmail.com
The following bug has been logged on the website:
Bug reference: 16866
Logged by: Ivan Salvato
Email address: ivan.salvato@gmail.com
PostgreSQL version: 12.0
Operating system: windows
Description:
Scenario
- vm Windows Server 2016
- PostgreSQL 12 (configurated wal_level = archive; archive_mode = on;
archive_command = 'copy "%p" "W:\\POSTGRES\\pg_log_archive\\%f" ')
- db 122 GB
- backup full of PostgreSQL with pg_basebackup (parameters -Ft -z) folder
W:\pg_backup. We delete the replicated transaction logs with
pg_archivecleanup in W:\pg_log_archive
Problem:
backup fails when the base.tar.gz achieve di 4 GB with pg_basebackup: error:
could not stat file
"W:\POSTGRES\pg_backup\Base-Backup_2021-02-05_103921/base.tar.gz": Unknown
error.
When we used Postgresql 9.6 it worked correctly.
Thanks in advance
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16866: pg_basebackup Windows Server 2016
2021-02-15 15:12 BUG #16866: pg_basebackup Windows Server 2016 PG Bug reporting form <noreply@postgresql.org>
@ 2021-02-16 01:00 ` Michael Paquier <michael@paquier.xyz>
2021-02-16 07:07 ` Re: BUG #16866: pg_basebackup Windows Server 2016 Ivan Salvato <ivan.salvato@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Michael Paquier @ 2021-02-16 01:00 UTC (permalink / raw)
To: ivan.salvato@gmail.com; pgsql-bugs@lists.postgresql.org
On Mon, Feb 15, 2021 at 03:12:10PM +0000, PG Bug reporting form wrote:
> Problem:
> backup fails when the base.tar.gz achieve di 4 GB with pg_basebackup: error:
> could not stat file
> "W:\POSTGRES\pg_backup\Base-Backup_2021-02-05_103921/base.tar.gz": Unknown
> error.
This comes from the fact that Windows stat() is not able to handle
files larger than 4GB by design, and the fact that pg_basebackup tries
to make durable all the contents of a base backup when it is done
streaming something on the target host. I bet that the failure you
are seeing is the fsync() part for base.tar.gz.
You can bypass that by using --no-sync as option, which would
basically emulate what Postgres <= 9.6 is doing. In Postgres 14, the
emulation of stat() has been fixed to handle the case of files larger
than 4GB:
https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=bed90759fcbcd72d4d06969eebab81e47326f...
This is arguably a bugfix, so we may consider a backpatch in the
future, but knowing how invasive the fix is we are still in a phase
where we want to have some dust settle on this change and Windows has
its own way to do weird things all the time. IMO, It may be better to
revisit that a couple of months after 14 is released so as there is
some feedback from the field with this change.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16866: pg_basebackup Windows Server 2016
2021-02-15 15:12 BUG #16866: pg_basebackup Windows Server 2016 PG Bug reporting form <noreply@postgresql.org>
2021-02-16 01:00 ` Re: BUG #16866: pg_basebackup Windows Server 2016 Michael Paquier <michael@paquier.xyz>
@ 2021-02-16 07:07 ` Ivan Salvato <ivan.salvato@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Ivan Salvato @ 2021-02-16 07:07 UTC (permalink / raw)
To: Michael Paquier <michael@paquier.xyz>; +Cc: pgsql-bugs@lists.postgresql.org
Thank you very much for your reply
Best Regards
Ivan
> On 16 Feb 2021, at 02:00, Michael Paquier <michael@paquier.xyz> wrote:
>
> On Mon, Feb 15, 2021 at 03:12:10PM +0000, PG Bug reporting form wrote:
>> Problem:
>> backup fails when the base.tar.gz achieve di 4 GB with pg_basebackup: error:
>> could not stat file
>> "W:\POSTGRES\pg_backup\Base-Backup_2021-02-05_103921/base.tar.gz": Unknown
>> error.
>
> This comes from the fact that Windows stat() is not able to handle
> files larger than 4GB by design, and the fact that pg_basebackup tries
> to make durable all the contents of a base backup when it is done
> streaming something on the target host. I bet that the failure you
> are seeing is the fsync() part for base.tar.gz.
>
> You can bypass that by using --no-sync as option, which would
> basically emulate what Postgres <= 9.6 is doing. In Postgres 14, the
> emulation of stat() has been fixed to handle the case of files larger
> than 4GB:
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=bed90759fcbcd72d4d06969eebab81e47326f...
>
> This is arguably a bugfix, so we may consider a backpatch in the
> future, but knowing how invasive the fix is we are still in a phase
> where we want to have some dust settle on this change and Windows has
> its own way to do weird things all the time. IMO, It may be better to
> revisit that a couple of months after 14 is released so as there is
> some feedback from the field with this change.
> --
> Michael
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16866: pg_basebackup Windows Server 20160
@ 2022-04-20 02:56 Takahiro Kitayama <kitayama@sraoss.co.jp>
2022-04-20 16:24 ` Re: BUG #16866: pg_basebackup Windows Server 20160 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 1 reply; 5+ messages in thread
From: Takahiro Kitayama @ 2022-04-20 02:56 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org
Hi,
PostgreSQL 14 has been fixed to handle files larger than 4GB on Windows,
but is not currently backported to previous versions.
In pgsql-bugs ML, considering this several months after the PostgreSQL 14 release 2021/9.
I would like to know about current status, and if there is no problem, please backport to past versions.
The excerpt below is an email about this matter.
Re: BUG #16866: pg_basebackup Windows Server 2016
https://www.postgresql.org/message-id/flat/YCsZIX2A2Ilsvfnl%40paquier.xyz#64134affb64b2ec286614648c2...
| From: Michael Paquier <michael(at)paquier(dot)xyz>
| To: ivan(dot)salvato(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org
| Subject: Re: BUG #16866: pg_basebackup Windows Server 2016
| Date: 2021-02-16 01:00:17
| Message-ID: YCsZIX2A2Ilsvfnl@paquier.xyz
| Views: Raw Message | Whole Thread | Download mbox | Resend email
| Lists: pgsql-bugs
|
| On Mon, Feb 15, 2021 at 03:12:10PM +0000, PG Bug reporting form wrote:
| > Problem:
| > backup fails when the base.tar.gz achieve di 4 GB with pg_basebackup: error:
| > could not stat file
| > "W:\POSTGRES\pg_backup\Base-Backup_2021-02-05_103921/base.tar.gz": Unknown
| > error.
|
|
| This comes from the fact that Windows stat() is not able to handle
| files larger than 4GB by design, and the fact that pg_basebackup tries
| to make durable all the contents of a base backup when it is done
| streaming something on the target host. I bet that the failure you
| are seeing is the fsync() part for base.tar.gz.
|
|
| You can bypass that by using --no-sync as option, which would
| basically emulate what Postgres <= 9.6 is doing. In Postgres 14, the
| emulation of stat() has been fixed to handle the case of files larger
| than 4GB:
| https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=bed90759fcbcd72d4d06969eebab81e47326f...
|
|
| This is arguably a bugfix, so we may consider a backpatch in the
| future, but knowing how invasive the fix is we are still in a phase
| where we want to have some dust settle on this change and Windows has
| its own way to do weird things all the time. IMO, It may be better to
| revisit that a couple of months after 14 is released so as there is
| some feedback from the field with this change.
| --
| Michael
Thanks in advance
--
Takahiro Kitayama <kitayama@sraoss.co.jp>
SRA OSS, Inc. Japan
http://www.sraoss.co.jp/
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: BUG #16866: pg_basebackup Windows Server 20160
2022-04-20 02:56 Re: BUG #16866: pg_basebackup Windows Server 20160 Takahiro Kitayama <kitayama@sraoss.co.jp>
@ 2022-04-20 16:24 ` Peter Eisentraut <peter.eisentraut@enterprisedb.com>
0 siblings, 0 replies; 5+ messages in thread
From: Peter Eisentraut @ 2022-04-20 16:24 UTC (permalink / raw)
To: Takahiro Kitayama <kitayama@sraoss.co.jp>; pgsql-bugs@lists.postgresql.org
On 20.04.22 04:56, Takahiro Kitayama wrote:
> PostgreSQL 14 has been fixed to handle files larger than 4GB on Windows,
> but is not currently backported to previous versions.
> In pgsql-bugs ML, considering this several months after the PostgreSQL 14 release 2021/9.
> I would like to know about current status, and if there is no problem, please backport to past versions.
I think that patch is too invasive for backpatching.
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2022-04-20 16:24 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-02-15 15:12 BUG #16866: pg_basebackup Windows Server 2016 PG Bug reporting form <noreply@postgresql.org>
2021-02-16 01:00 ` Michael Paquier <michael@paquier.xyz>
2021-02-16 07:07 ` Ivan Salvato <ivan.salvato@gmail.com>
2022-04-20 02:56 Re: BUG #16866: pg_basebackup Windows Server 20160 Takahiro Kitayama <kitayama@sraoss.co.jp>
2022-04-20 16:24 ` Re: BUG #16866: pg_basebackup Windows Server 20160 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox