pg.ddx.io  pgbackrest/pgbackrest GitHub issues and pull requests  
help / color / mirror / Atom feed
From: fmbiete (@fmbiete) <noreply+fmbiete@github.com>
To: pgbackrest/pgbackrest <noreply+pgbackrest-pgbackrest@github.com>
Subject: Re: [pgbackrest/pgbackrest] PR #2887: DirectIO
Date: Sat, 31 Oct 2026 13:29:30 +0000
Message-ID: <gh-pgbackrest-pgbackrest-2887-c6543992364@github.com> (raw)
In-Reply-To: <gh-pgbackrest-pgbackrest-2887@github.com>
References: <gh-pgbackrest-pgbackrest-2887@github.com>

Use DirectIO to avoid polluting buffer cache with reads or writes from pgbackrest.

Calling fadvise POSIX_FADV_DONTNEED after the read purges the cached pages for that file, it doesn't avoid caching them in the first place. That means we don't avoid polluting the cache and we could evict PostgreSQL cached pages.

Example (test01.c) - you can see it has fully cached the read file and fadvise evicts the pages.
```
Creating 1GB file via O_DIRECT...
[BEFORE READ] Cached pages: 0 / 262144 (Size: 0.00 MB)
Reading file sequentially...
Read 1073741824 bytes. Execution time: 2019.85 ms
[AFTER READ ] Cached pages: 262144 / 262144 (Size: 1024.00 MB)
Calling posix_fadvise(POSIX_FADV_DONTNEED) to evict cache...
[AFTER EVICTION] Cached pages: 0 / 262144 (Size: 0.00 MB)
```


Calling fadvise before reading with POSIX_FADV_NOREUSE should be the way to go, but the Linux implementation is not what we would expect, or we don't see the benefit with 1 file test.

Example (test02.c) - kernel 7.2
```
Creating 1GB file via O_DIRECT...
[BEFORE READ] Cached pages: 0 / 262144 (Size: 0.00 MB)
Calling posix_fadvise(POSIX_FADV_NOREUSE)...
Reading file sequentially...
Read 1073741824 bytes. Execution time: 2018.80 ms
[AFTER READ ] Cached pages: 262144 / 262144 (Size: 1024.00 MB)
Calling posix_fadvise(POSIX_FADV_DONTNEED) to evict cache...
[AFTER EVICTION] Cached pages: 0 / 262144 (Size: 0.00 MB)
```

Using DirectIO avoids the page pollution, of course it is slower than cached IO: 1.27x in my NVME. Would that be a deal breaker for pgbackrest? I don't think so, and it avoids the above issues.
```
Creating 1GB file via O_DIRECT...
[BEFORE READ] Cached pages: 0 / 262144 (Size: 0.00 MB)
Opening file with O_DIRECT...
Reading file sequentially via O_DIRECT...
Read 1073741824 bytes. Execution time: 2555.02 ms
[AFTER READ ] Cached pages: 0 / 262144 (Size: 0.00 MB)
```



If you think this is adequate I could do the changes you commented and avoid using vscode auto-formatter on-save, because code-format doesn't seem to fix the breaks.

[test01.c](https://github.com/user-attachments/files/34217940/test01.c)
[test02.c](https://github.com/user-attachments/files/34217941/test02.c)
[test03.c](https://github.com/user-attachments/files/34217942/test03.c)


view thread (20+ messages)  latest in thread

Message-ID: <gh-pgbackrest-pgbackrest-2887-c6543992364@github.com>
Permalink:  ../gh-pgbackrest-pgbackrest-2887-c6543992364@github.com/

 · 


This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox