pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedxfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
9+ messages / 5 participants
[nested] [flat]
* xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
@ 2012-12-05 15:34 Andrea Suisani <sickpig@opinioni.net>
2012-12-05 16:51 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Jean-David Beyer <jeandavid8@verizon.net>
2012-12-06 08:29 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
0 siblings, 2 replies; 9+ messages in thread
From: Andrea Suisani @ 2012-12-05 15:34 UTC (permalink / raw)
To: pgsql-performance
[sorry for resuming an old thread]
[cut]
>>> Question is... will that remove the performance penalty of HyperThreading?
>>
>> So I've added to my todo list to perform a test to verify this claim :)
>
> done.
on this box:
> in a brief: the box is dell a PowerEdge r720 with 16GB of RAM,
> the cpu is a Xeon 5620 with 6 core, the OS is installed on a raid
> (sata disk 7.2k rpm) and the PGDATA is on separate RAID 1 array
> (sas 15K rpm) and the controller is a PERC H710 (bbwc with a cache
> of 512 MB). (ubuntu 12.04)
with postgres 9.2.1 and $PGDATA on a ext4 formatted partition
i've got:
> those are the results:
>
> HT HT SYSFS DIS HT BIOS DISABLE
> -c -t r1 r2 r3 r1 r2 r3 r1 r2 r3
> 5 20K 1641 1831 1496 2020 1974 2033 2005 1988 1967
> 10 10K 2161 2134 2136 2277 2252 2216 1854 1824 1810
> 20 5k 2550 2508 2558 2417 2388 2357 1924 1928 1954
> 30 3333 2216 2272 2250 2333 2493 2496 1993 2009 2008
> 40 2.5K 2179 2221 2250 2568 2535 2500 2025 2048 2018
> 50 2K 2217 2213 2213 2487 2449 2604 2112 2016 2023
on the same machine with the same configuration,
having PGDATA on a xfs formatted partition gives me
a much better TPS.
e.g. pgbench -c 20 -t 5000 gives me 6305 TPS
(3 runs with "echo 3 > /proc/sys/vm/drop_caches && /etc/init.d/postgresql-9.2 restart"
in between).
Anybody else have experienced this kind of differences
between etx4 and xfs?
Andrea
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
@ 2012-12-05 16:51 ` Jean-David Beyer <jeandavid8@verizon.net>
2012-12-05 16:56 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Claudio Freire <klaussfreire@gmail.com>
2012-12-05 17:00 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrew Dunstan <andrew@dunslane.net>
1 sibling, 2 replies; 9+ messages in thread
From: Jean-David Beyer @ 2012-12-05 16:51 UTC (permalink / raw)
To: pgsql-performance
On 12/05/2012 10:34 AM, Andrea Suisani wrote:
> [sorry for resuming an old thread]
>
> [cut]
>
>>>> Question is... will that remove the performance penalty of
>>>> HyperThreading?
>>>
>>> So I've added to my todo list to perform a test to verify this claim :)
>>
>> done.
>
> on this box:
>
>> in a brief: the box is dell a PowerEdge r720 with 16GB of RAM,
>> the cpu is a Xeon 5620 with 6 core, the OS is installed on a raid
>> (sata disk 7.2k rpm) and the PGDATA is on separate RAID 1 array
>> (sas 15K rpm) and the controller is a PERC H710 (bbwc with a cache
>> of 512 MB). (ubuntu 12.04)
>
> with postgres 9.2.1 and $PGDATA on a ext4 formatted partition
> i've got:
>
>> those are the results:
>>
>> HT HT SYSFS DIS HT BIOS DISABLE
>> -c -t r1 r2 r3 r1 r2 r3 r1 r2 r3
>> 5 20K 1641 1831 1496 2020 1974 2033 2005 1988 1967
>> 10 10K 2161 2134 2136 2277 2252 2216 1854 1824 1810
>> 20 5k 2550 2508 2558 2417 2388 2357 1924 1928 1954
>> 30 3333 2216 2272 2250 2333 2493 2496 1993 2009 2008
>> 40 2.5K 2179 2221 2250 2568 2535 2500 2025 2048 2018
>> 50 2K 2217 2213 2213 2487 2449 2604 2112 2016 2023
>
> on the same machine with the same configuration,
> having PGDATA on a xfs formatted partition gives me
> a much better TPS.
>
> e.g. pgbench -c 20 -t 5000 gives me 6305 TPS
> (3 runs with "echo 3 > /proc/sys/vm/drop_caches &&
> /etc/init.d/postgresql-9.2 restart"
> in between).
>
> Anybody else have experienced this kind of differences
> between etx4 and xfs?
>
> Andrea
>
>
>
I thought that postgreSQL did its own journalling, if that is the proper
term, so why not use an ext2 file system to lower overhead?
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-05 16:51 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Jean-David Beyer <jeandavid8@verizon.net>
@ 2012-12-05 16:56 ` Claudio Freire <klaussfreire@gmail.com>
1 sibling, 0 replies; 9+ messages in thread
From: Claudio Freire @ 2012-12-05 16:56 UTC (permalink / raw)
To: Jean-David Beyer <jeandavid8@verizon.net>; +Cc: pgsql-performance
On Wed, Dec 5, 2012 at 1:51 PM, Jean-David Beyer <jeandavid8@verizon.net> wrote:
> I thought that postgreSQL did its own journalling, if that is the proper
> term, so why not use an ext2 file system to lower overhead?
Because you can still have metadata-level corruption.
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-05 16:51 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Jean-David Beyer <jeandavid8@verizon.net>
@ 2012-12-05 17:00 ` Andrew Dunstan <andrew@dunslane.net>
1 sibling, 0 replies; 9+ messages in thread
From: Andrew Dunstan @ 2012-12-05 17:00 UTC (permalink / raw)
To: Jean-David Beyer <jeandavid8@verizon.net>; +Cc: pgsql-performance
On 12/05/2012 11:51 AM, Jean-David Beyer wrote:
>>
>>
> I thought that postgreSQL did its own journalling, if that is the
> proper term, so why not use an ext2 file system to lower overhead?
Postgres journalling will not save you from a corrupt file system.
cheers
andrew
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
@ 2012-12-06 08:29 ` John Lister <john.lister@kickstone.com>
2012-12-06 08:44 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
1 sibling, 1 reply; 9+ messages in thread
From: John Lister @ 2012-12-06 08:29 UTC (permalink / raw)
To: Andrea Suisani <sickpig@opinioni.net>; pgsql-performance
> on this box:
>
>> in a brief: the box is dell a PowerEdge r720 with 16GB of RAM,
>> the cpu is a Xeon 5620 with 6 core, the OS is installed on a raid
>> (sata disk 7.2k rpm) and the PGDATA is on separate RAID 1 array
>> (sas 15K rpm) and the controller is a PERC H710 (bbwc with a cache
>> of 512 MB). (ubuntu 12.04)
>
> on the same machine with the same configuration,
> having PGDATA on a xfs formatted partition gives me
> a much better TPS.
>
> e.g. pgbench -c 20 -t 5000 gives me 6305 TPS
> (3 runs with "echo 3 > /proc/sys/vm/drop_caches &&
> /etc/init.d/postgresql-9.2 restart"
> in between).
Hi, I found this interesting as I'm trying to do some benchmarks on my
box which is very similar to the above but I don't believe the tps is
any where near what it should be. Is the 6305 figure from xfs? I'm
assuming that your main data array is just 2 15k sas drives, are you
putting the WAL on the data array or is that stored somewhere else? Can
I ask what scaling params, etc you used to build the pgbench tables and
look at your postgresql.conf file to see if I missed something (offline
if you wish)
I'm running 8x SSDs in RAID 10 for the data and pull just under 10k on a
xfs system which is much lower than I'd expect for that setup and isn't
significantly greater than your reported results, so something must be
very wrong.
Thanks
John
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 08:29 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
@ 2012-12-06 08:44 ` Andrea Suisani <sickpig@opinioni.net>
2012-12-06 09:33 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
0 siblings, 1 reply; 9+ messages in thread
From: Andrea Suisani @ 2012-12-06 08:44 UTC (permalink / raw)
To: John Lister <john.lister@kickstone.com>; +Cc: pgsql-performance
Hi John,
On 12/06/2012 09:29 AM, John Lister wrote:
>
>> on this box:
>>
>>> in a brief: the box is dell a PowerEdge r720 with 16GB of RAM,
>>> the cpu is a Xeon 5620 with 6 core, the OS is installed on a raid
>>> (sata disk 7.2k rpm) and the PGDATA is on separate RAID 1 array
>>> (sas 15K rpm) and the controller is a PERC H710 (bbwc with a cache
>>> of 512 MB). (ubuntu 12.04)
>>
>> on the same machine with the same configuration,
>> having PGDATA on a xfs formatted partition gives me
>> a much better TPS.
>>
>> e.g. pgbench -c 20 -t 5000 gives me 6305 TPS
>> (3 runs with "echo 3 > /proc/sys/vm/drop_caches && /etc/init.d/postgresql-9.2 restart"
>> in between).
> Hi, I found this interesting as I'm trying to do some benchmarks on my box which is
> very similar to the above but I don't believe the tps is any where near what it should be.
> Is the 6305 figure from xfs?
yes, it is.
> I'm assuming that your main data array is just 2 15k sas drives,
correct
> are you putting the WAL on the data array or is that stored somewhere else?
pg_xlog is placed in the data array.
> Can I ask what scaling params,
sure, I've initialized pgbench db issuing:
pgbench -i -s 10 pgbench
> etc you used to build the pgbench tables and look at your postgresql.conf file to see if I missed something (offline if you wish)
those are non default values in postgresql.conf
listen_addresses = '*'
max_connections = 100
shared_buffers = 3200MB
work_mem = 30MB
maintenance_work_mem = 800MB
synchronous_commit = off
full_page_writes = off
checkpoint_segments = 40
checkpoint_completion_target = 0.9
random_page_cost = 3.5
effective_cache_size = 10GB
log_timezone = 'localtime'
stats_temp_directory = 'pg_stat_tmp_ram'
autovacuum_naptime = 5min
and then OS tweaks:
HT bios disabled
/sbin/blockdev --setra 8192 /dev/sdb
echo deadline > /sys/block/sdb/queue/scheduler
vm.swappiness=0
vm.overcommit_memory=2
vm.dirty_ratio=2
vm.dirty_background_ratio=1
kernel.shmmax=3454820352
kernel.shmall=2048341
$PGDATA is on xfs (rw,noatime)
tmpfs on /db/9.2/pg_stat_tmp_ram type tmpfs (rw,size=50M,uid=1001,gid=1001)
kernel 3.2.0-32-generic
Andrea
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 08:29 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
2012-12-06 08:44 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
@ 2012-12-06 09:33 ` Andrea Suisani <sickpig@opinioni.net>
2012-12-06 11:37 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
0 siblings, 1 reply; 9+ messages in thread
From: Andrea Suisani @ 2012-12-06 09:33 UTC (permalink / raw)
To: John Lister <john.lister@kickstone.com>; +Cc: pgsql-performance
[added performance list back]
On 12/06/2012 10:04 AM, John Lister wrote:
> Thanks for the info, I'll have a play and see what values I get with similar settings, etc
you're welcome
> Still think something is wrong with my config, but we'll see.
which kind of ssd disks do you have ?
maybe they are of the same typeShaun Thomas is having problem with here:
http://archives.postgresql.org/pgsql-performance/2012-12/msg00030.php
Andrea
> john
>
> On 06/12/2012 08:44, Andrea Suisani wrote:
>> Hi John,
>>
>> On 12/06/2012 09:29 AM, John Lister wrote:
>>>
>>>> on this box:
>>>>
>>>>> in a brief: the box is dell a PowerEdge r720 with 16GB of RAM,
>>>>> the cpu is a Xeon 5620 with 6 core, the OS is installed on a raid
>>>>> (sata disk 7.2k rpm) and the PGDATA is on separate RAID 1 array
>>>>> (sas 15K rpm) and the controller is a PERC H710 (bbwc with a cache
>>>>> of 512 MB). (ubuntu 12.04)
>>>>
>>>> on the same machine with the same configuration,
>>>> having PGDATA on a xfs formatted partition gives me
>>>> a much better TPS.
>>>>
>>>> e.g. pgbench -c 20 -t 5000 gives me 6305 TPS
>>>> (3 runs with "echo 3 > /proc/sys/vm/drop_caches && /etc/init.d/postgresql-9.2 restart"
>>>> in between).
>>
>>
>>> Hi, I found this interesting as I'm trying to do some benchmarks on my box which is
>> > very similar to the above but I don't believe the tps is any where near what it should be.
>> > Is the 6305 figure from xfs?
>>
>> yes, it is.
>>
>>> I'm assuming that your main data array is just 2 15k sas drives,
>>
>> correct
>>
>>> are you putting the WAL on the data array or is that stored somewhere else?
>>
>> pg_xlog is placed in the data array.
>>
>>> Can I ask what scaling params,
>>
>> sure, I've initialized pgbench db issuing:
>>
>> pgbench -i -s 10 pgbench
>>
>>> etc you used to build the pgbench tables and look at your postgresql.conf file to see if I missed something (offline if you wish)
>>
>> those are non default values in postgresql.conf
>>
>> listen_addresses = '*'
>> max_connections = 100
>> shared_buffers = 3200MB
>> work_mem = 30MB
>> maintenance_work_mem = 800MB
>> synchronous_commit = off
>> full_page_writes = off
>> checkpoint_segments = 40
>> checkpoint_completion_target = 0.9
>> random_page_cost = 3.5
>> effective_cache_size = 10GB
>> log_timezone = 'localtime'
>> stats_temp_directory = 'pg_stat_tmp_ram'
>> autovacuum_naptime = 5min
>>
>> and then OS tweaks:
>>
>> HT bios disabled
>> /sbin/blockdev --setra 8192 /dev/sdb
>> echo deadline > /sys/block/sdb/queue/scheduler
>> vm.swappiness=0
>> vm.overcommit_memory=2
>> vm.dirty_ratio=2
>> vm.dirty_background_ratio=1
>> kernel.shmmax=3454820352
>> kernel.shmall=2048341
>> $PGDATA is on xfs (rw,noatime)
>> tmpfs on /db/9.2/pg_stat_tmp_ram type tmpfs (rw,size=50M,uid=1001,gid=1001)
>> kernel 3.2.0-32-generic
>>
>>
>> Andrea
>>
>>
>
>
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 08:29 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
2012-12-06 08:44 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 09:33 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
@ 2012-12-06 11:37 ` John Lister <john.lister@kickstone.com>
2012-12-06 12:53 ` Re: Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
0 siblings, 1 reply; 9+ messages in thread
From: John Lister @ 2012-12-06 11:37 UTC (permalink / raw)
To: Andrea Suisani <sickpig@opinioni.net>; +Cc: pgsql-performance
On 06/12/2012 09:33, Andrea Suisani wrote:
>
> which kind of ssd disks do you have ?
> maybe they are of the same typeShaun Thomas is having problem with here:
> http://archives.postgresql.org/pgsql-performance/2012-12/msg00030.php
Yeah i saw that post, I'm running the same version of ubuntu with the
3.2 kernel, so when I get a chance to take it down will try the new
kernels, although ubuntu are on 3.5 now... Shaun didn't post what
hardware he was running on, so it would be interesting to see how it
compares. They are intel 320s, which while not the newest should offer
some protection against power failure, etc
John
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance]
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 08:29 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
2012-12-06 08:44 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 09:33 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-06 11:37 ` Re: xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] John Lister <john.lister@kickstone.com>
@ 2012-12-06 12:53 ` Andrea Suisani <sickpig@opinioni.net>
0 siblings, 0 replies; 9+ messages in thread
From: Andrea Suisani @ 2012-12-06 12:53 UTC (permalink / raw)
To: John Lister <john.lister@kickstone.com>; +Cc: pgsql-performance
On 12/06/2012 12:37 PM, John Lister wrote:
> On 06/12/2012 09:33, Andrea Suisani wrote:
>>
>> which kind of ssd disks do you have ?
>> maybe they are of the same typeShaun Thomas is having problem with here:
>> http://archives.postgresql.org/pgsql-performance/2012-12/msg00030.php
> Yeah i saw that post, I'm running the same version of ubuntu with the 3.2 kernel, so when I get a chance to take it down will try the new kernels, although ubuntu are on 3.5 now... Shaun didn't post what hardware he was running on, so it would be interesting to see how it compares. They are intel
> 320s, which while not the newest should offer some protection against power failure, etc
reading again the thread I realized Shaun is using
fusionIO driver and he said that the regression is due
to "some recent 3.2 kernel patch borks the driver in
some horrible way".
so maybe you're not on the same boat (since you're
using intel 320), or maybe the kernel regression
he's referring to is related to the kernel subsystem
that deal with ssd disks independently from brands.
In the latter case testing a different kernel would be worthy.
Andrea
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2012-12-06 12:53 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-12-05 15:34 xfs perform a lot better than ext4 [WAS: Re: Two identical systems, radically different performance] Andrea Suisani <sickpig@opinioni.net>
2012-12-05 16:51 ` Jean-David Beyer <jeandavid8@verizon.net>
2012-12-05 16:56 ` Claudio Freire <klaussfreire@gmail.com>
2012-12-05 17:00 ` Andrew Dunstan <andrew@dunslane.net>
2012-12-06 08:29 ` John Lister <john.lister@kickstone.com>
2012-12-06 08:44 ` Andrea Suisani <sickpig@opinioni.net>
2012-12-06 09:33 ` Andrea Suisani <sickpig@opinioni.net>
2012-12-06 11:37 ` John Lister <john.lister@kickstone.com>
2012-12-06 12:53 ` Andrea Suisani <sickpig@opinioni.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox