pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feed8Kb or 4Kb ext4 filesystem page size
5+ messages / 5 participants
[nested] [flat]
* 8Kb or 4Kb ext4 filesystem page size
@ 2019-05-29 18:10 Alexandre hadjinlian guerra <alexhguerra@gmail.com>
2019-05-30 07:12 ` Re: 8Kb or 4Kb ext4 filesystem page size Achilleas Mantzios <achill@matrix.gatewaynet.com>
2019-06-01 19:27 ` 8Kb or 4Kb ext4 filesystem page size Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
0 siblings, 2 replies; 5+ messages in thread
From: Alexandre hadjinlian guerra @ 2019-05-29 18:10 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
Hi
Given that postgres uses 8Kb pages, im wondering why i couldnt see any
tests at all which would format ext4 partition to 8Kb pages. Im about to do
some tests, but any knowledge about such lack of tests on the internet
makes me wonder if im looking poorly or just lack of testing. besides, i do
ask if the following link remain true given XFS and EXT4 evolution since
2015
https://blog.pgaddict.com/posts/postgresql-performance-on-ext4-and-xfs
Thanks
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: 8Kb or 4Kb ext4 filesystem page size
2019-05-29 18:10 8Kb or 4Kb ext4 filesystem page size Alexandre hadjinlian guerra <alexhguerra@gmail.com>
@ 2019-05-30 07:12 ` Achilleas Mantzios <achill@matrix.gatewaynet.com>
2019-05-31 13:35 ` Re: 8Kb or 4Kb ext4 filesystem page size tutaver@tuta.io
1 sibling, 1 reply; 5+ messages in thread
From: Achilleas Mantzios @ 2019-05-30 07:12 UTC (permalink / raw)
To: pgsql-admin@lists.postgresql.org
On 29/5/19 9:10 μ.μ., Alexandre hadjinlian guerra wrote:
> Hi
>
> Given that postgres uses 8Kb pages, im wondering why i couldnt see any tests at all which would format ext4 partition to 8Kb pages. Im about to do some tests, but any knowledge about such lack of
> tests on the internet makes me wonder if im looking poorly or just lack of testing. besides, i do ask if the following link remain true given XFS and EXT4 evolution since 2015
> https://blog.pgaddict.com/posts/postgresql-performance-on-ext4-and-xfs
This is just my personal opinion, and might be the case only with us, but anytime I had severe problems with any of our postgresql clusters this was with XFS. I know that XFS has been highly praised
and recommended as a pgsql fs, just hasn't been the case with us.
Having said that, in the era of could based or vm-based postgresql installation, the question of choice of FS is becoming less and less relevant. Just make sure to secure your data, and that you don't
over tune your fs at the expense of data safety.
>
> Thanks
--
Achilleas Mantzios
IT DEV Lead
IT DEPT
Dynacom Tankers Mgmt
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: 8Kb or 4Kb ext4 filesystem page size
2019-05-29 18:10 8Kb or 4Kb ext4 filesystem page size Alexandre hadjinlian guerra <alexhguerra@gmail.com>
2019-05-30 07:12 ` Re: 8Kb or 4Kb ext4 filesystem page size Achilleas Mantzios <achill@matrix.gatewaynet.com>
@ 2019-05-31 13:35 ` tutaver@tuta.io
0 siblings, 0 replies; 5+ messages in thread
From: tutaver@tuta.io @ 2019-05-31 13:35 UTC (permalink / raw)
To: Achilleas Mantzios <achill@matrix.gatewaynet.com>; +Cc: pgsql-admin@lists.postgresql.org
Hello,
Great article! "FS is becoming less and less relevant", agreed. For the days of raw...
What about when it comes to postgres file system design? By this I mean I've been challenged to find a good resource addressing postgres file system best practice regardless of EXT4, XFS, etc.. For example, partition order on drive, reordering, tuning and more. Question is obviously dependent upon OS' available file systems, just thought I'd ask what others are coming across.
Thanks
--
Securely sent with Tutanota. Get your own encrypted, ad-free mailbox:
https://tutanota.com
May 30, 2019, 1:12 AM by achill@matrix.gatewaynet.com:
> On 29/5/19 9:10 μ.μ., Alexandre hadjinlian guerra wrote:
>
>> Hi
>>
>> Given that postgres uses 8Kb pages, im wondering why i couldnt see any tests at all which would format ext4 partition to 8Kb pages. Im about to do some tests, but any knowledge about such lack of tests on the internet makes me wonder if im looking poorly or just lack of testing. besides, i do ask if the following link remain true given XFS and EXT4 evolution since 2015
>> https://blog.pgaddict.com/posts/postgresql-performance-on-ext4-and-xfs
>>
> This is just my personal opinion, and might be the case only with us, but anytime I had severe problems with any of our postgresql clusters this was with XFS. I know that XFS has been highly praised and recommended as a pgsql fs, just hasn't been the case with us.
> Having said that, in the era of could based or vm-based postgresql installation, the question of choice of FS is becoming less and less relevant. Just make sure to secure your data, and that you don't over tune your fs at the expense of data safety.
>
>>
>> Thanks
>>
>
>
> --
> Achilleas Mantzios
> IT DEV Lead
> IT DEPT
> Dynacom Tankers Mgmt
>
^ permalink raw reply [nested|flat] 5+ messages in thread
* 8Kb or 4Kb ext4 filesystem page size
2019-05-29 18:10 8Kb or 4Kb ext4 filesystem page size Alexandre hadjinlian guerra <alexhguerra@gmail.com>
@ 2019-06-01 19:27 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
1 sibling, 0 replies; 5+ messages in thread
From: Peter Eisentraut @ 2019-06-01 19:27 UTC (permalink / raw)
To: Alexandre hadjinlian guerra <alexhguerra@gmail.com>; pgsql-admin@lists.postgresql.org
On 2019-05-29 14:10, Alexandre hadjinlian guerra wrote:
> Given that postgres uses 8Kb pages, im wondering why i couldnt see any
> tests at all which would format ext4 partition to 8Kb pages.
I suppose this is because the maximum block size for ext4 is 4 kB.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: 8Kb or 4Kb ext4 filesystem page size
@ 2019-06-02 07:29 George Neuner <gneuner2@comcast.net>
0 siblings, 0 replies; 5+ messages in thread
From: George Neuner @ 2019-06-02 07:29 UTC (permalink / raw)
To: pgsql-admin
On Wed, 29 May 2019 15:10:06 -0300, Alexandre hadjinlian guerra
<alexhguerra@gmail.com> wrote:
> Hi
>
> Given that postgres uses 8Kb pages, im wondering why i couldnt see any
> tests at all which would format ext4 partition to 8Kb pages. Im about
> to do
> some tests, but any knowledge about such lack of tests on the internet
> makes me wonder if im looking poorly or just lack of testing. besides,
> i do
> ask if the following link remain true given XFS and EXT4 evolution since
> 2015
> https://blog.pgaddict.com/posts/postgresql-performance-on-ext4-and-xfs
>
> Thanks
One thing no one has yet mentioned is that I/O performance could suffer
greatly if the disk page size > memory page size because a single disk
page split over multiple VMM page frames may be discontiguous in memory.
That is a problem for bus-mastering disk controllers because their DMA
can operate only on *physical* addresses - not the logical addresses
used by the programs. Pages touched by external DMA need to be pinned
(locked in place) for the duration.
There is also DMA built-in on the system board. Typically built-in DMA
can work through the MMU with logical addresses and so (usually) does
not need to pin memory pages to access them.
But it's up to the device driver which DMA (if any) is used. Most
bus-mastering devices prefer to use their own DMA hardware, and their
drivers either have to pin memory pages or work through a small(ish)
buffer in a fixed location (which entails extraneous copying of data).
Postgresql's 8KB logical disk pages take up two 4KB memory pages - which
may not be adjacent - but since the filesystem and memory pages are the
same size, DMA (built-in or external) can access the pages in any
order, and without employing (or even needing) scatter/gather ability to
coalesce or distribute partial pages to/from non-contiguous locations.
Another consideration for disk page size is that program code typically
is paged in directly from the executable file. If the disk and memory
pages aren't the same size, the OS page fault handler needs to be aware
and able to deal with the difference. Obviously this could be addressed
simply by segregating "large" pages to separate data-only volumes.
AFAIK, only the Itanium has an option for 8KB memory pages. The "large"
/ "huge" memory pages available in most CPUs today are too big to be
used effectively by a filesystem.
https://en.wikipedia.org/wiki/Page_(computer_memory)#Multiple_page_sizes
Rewriting filesystem drivers and the memory manager so that 8KKB or
larger disk pages could be treated as a sort of "huge" memory page -
overlaid on adjacent 4KB physical memory pages - would be a massive
job. Since few programs other than DBMS really would benefit from it,
it isn't likely to happen.
YMMV,
George
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2019-06-02 07:29 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-05-29 18:10 8Kb or 4Kb ext4 filesystem page size Alexandre hadjinlian guerra <alexhguerra@gmail.com>
2019-05-30 07:12 ` Achilleas Mantzios <achill@matrix.gatewaynet.com>
2019-05-31 13:35 ` tutaver@tuta.io
2019-06-01 19:27 ` Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2019-06-02 07:29 Re: 8Kb or 4Kb ext4 filesystem page size George Neuner <gneuner2@comcast.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox