public inbox for [email protected]
help / color / mirror / Atom feed[PATCH 13/18] doc review for FDW bulk inserts
3+ messages / 3 participants
[nested] [flat]
* [PATCH 13/18] doc review for FDW bulk inserts
@ 2021-01-23 23:17 Justin Pryzby <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Justin Pryzby @ 2021-01-23 23:17 UTC (permalink / raw)
b663a4136331de6c7364226e3dbf7c88bfee7145
---
doc/src/sgml/fdwhandler.sgml | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml
index 854913ae5f..12e00bfc2f 100644
--- a/doc/src/sgml/fdwhandler.sgml
+++ b/doc/src/sgml/fdwhandler.sgml
@@ -672,9 +672,8 @@ GetForeignModifyBatchSize(ResultRelInfo *rinfo);
Report the maximum number of tuples that a single
<function>ExecForeignBatchInsert</function> call can handle for
- the specified foreign table. That is, The executor passes at most
- the number of tuples that this function returns to
- <function>ExecForeignBatchInsert</function>.
+ the specified foreign table. The executor passes at most
+ the given number of tuples to <function>ExecForeignBatchInsert</function>.
<literal>rinfo</literal> is the <structname>ResultRelInfo</structname> struct describing
the target foreign table.
The FDW is expected to provide a foreign server and/or foreign
--
2.17.0
--lc9FT7cWel8HagAv
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename="0014-doc-review-logical-decode-in-prepare.patch"
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: FileFallocate misbehaving on XFS
@ 2024-12-09 10:06 Tomas Vondra <[email protected]>
2024-12-09 23:00 ` Re: FileFallocate misbehaving on XFS Michael Harris <[email protected]>
0 siblings, 1 reply; 3+ messages in thread
From: Tomas Vondra @ 2024-12-09 10:06 UTC (permalink / raw)
To: Michael Harris <[email protected]>; pgsql-hackers
On 12/9/24 08:34, Michael Harris wrote:
> Hello PG Hackers
>
> Our application has recently migrated to PG16, and we have experienced
> some failed upgrades. The upgrades are performed using pg_upgrade and
> have failed during the phase where the schema is restored into the new
> cluster, with the following error:
>
> pg_restore: error: could not execute query: ERROR: could not extend
> file "pg_tblspc/16401/PG_16_202307071/17643/1249.1" with
> FileFallocate(): No space left on device
> HINT: Check free disk space.
>
> This has happened multiple times on different servers, and in each
> case there was plenty of free space available.
>
> We found this thread describing similar issues:
>
> https://www.postgresql.org/message-id/flat/AS1PR05MB91059AC8B525910A5FCD6E699F9A2%40AS1PR05MB9105.eu...
>
> As is the case in that thread, all of the affected databases are using XFS.
>
> One of my colleagues built postgres from source with
> HAVE_POSIX_FALLOCATE not defined, and using that build he was able to
> complete the pg_upgrade, and then switched to a stock postgres build
> after the upgrade. However, as you might expect, after the upgrade we
> have experienced similar errors during regular operation. We make
> heavy use of COPY, which is mentioned in the above discussion as
> pre-allocating files.
>
> We have seen this on both Rocky Linux 8 (kernel 4.18.0) and Rocky
> Linux 9 (Kernel 5.14.0).
>
> I am wondering if this bug might be related:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1791323
>
>> When given an offset of 0 and a length, fallocate (man 2 fallocate) reports ENOSPC if the size of the file + the length to be allocated is greater than the available space.
>
> There is a reproduction procedure at the bottom of the above ubuntu
> thread, and using that procedure I get the same results on both kernel
> 4.18.0 and 5.14.0.
> When calling fallocate with offset zero on an existing file, I get
> enospc even if I am only requesting the same amount of space as the
> file already has.
> If I repeat the experiment with ext4 I don't get that behaviour.
>
> On a surface examination of the code paths leading to the
> FileFallocate call, it does not look like it should be trying to
> allocate already allocated space, but I might have missed something
> there.
>
> Is this already being looked into?
>
Sounds more like an XFS bug/behavior, so it's not clear to me what we
could do about it. I mean, if the filesystem reports bogus out-of-space,
is there even something we can do?
What is not clear to me is why would this affect pg_upgrade at all. We
have the data files split into 1GB segments, and the copy/clone/... goes
one by one. So there shouldn't be more than 1GB "extra" space needed.
Surely you have more free space on the system?
regards
--
Tomas Vondra
^ permalink raw reply [nested|flat] 3+ messages in thread
* Re: FileFallocate misbehaving on XFS
2024-12-09 10:06 Re: FileFallocate misbehaving on XFS Tomas Vondra <[email protected]>
@ 2024-12-09 23:00 ` Michael Harris <[email protected]>
0 siblings, 0 replies; 3+ messages in thread
From: Michael Harris @ 2024-12-09 23:00 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: pgsql-hackers
Hi Tomas
On Mon, 9 Dec 2024 at 21:06, Tomas Vondra <[email protected]> wrote:
> Sounds more like an XFS bug/behavior, so it's not clear to me what we
> could do about it. I mean, if the filesystem reports bogus out-of-space,
> is there even something we can do?
I don't disagree that it's most likely an XFS issue. However, XFS is
pretty widely used - it's the default FS for RHEL & the default in
SUSE for non-root partitions - so maybe some action should be taken.
Some things we could consider:
- Providing a way to configure PG not to use posix_fallocate at runtime
- Detecting the use of XFS (probably nasty and complex to do in a
platform independent way) and disable posix_fallocate
- In the case of posix_fallocate failing with ENOSPC, fall back to
FileZero (worst case that will fail as well, in which case we will
know that we really are out of space)
- Documenting that XFS might not be a good choice, at least for some
kernel versions
> What is not clear to me is why would this affect pg_upgrade at all. We
> have the data files split into 1GB segments, and the copy/clone/... goes
> one by one. So there shouldn't be more than 1GB "extra" space needed.
> Surely you have more free space on the system?
Yes, that also confused me. It actually fails during the schema
restore phase - where pg_upgrade calls pg_restore to restore a
schema-only dump that it takes earlier in the process. At this stage
it is only trying to restore the schema, not any actual table data.
Note that we use the --link option to pg_upgrade, so it should not be
using much space even when the table data is being upgraded.
The filesystems have >1TB free space when this has occurred.
It does continue to give this error after the upgrade, at apparently
random intervals, when data is being loaded into the DB using COPY
commands, so it might be best not to focus too much on the fact that
we first encounter it during the upgrade.
Cheers
Mike.
^ permalink raw reply [nested|flat] 3+ messages in thread
end of thread, other threads:[~2024-12-09 23:00 UTC | newest]
Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-01-23 23:17 [PATCH 13/18] doc review for FDW bulk inserts Justin Pryzby <[email protected]>
2024-12-09 10:06 Re: FileFallocate misbehaving on XFS Tomas Vondra <[email protected]>
2024-12-09 23:00 ` Re: FileFallocate misbehaving on XFS Michael Harris <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox