pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedSome performance testing?
21+ messages / 7 participants
[nested] [flat]
* Some performance testing?
@ 2015-03-31 19:52 Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
0 siblings, 1 reply; 21+ messages in thread
From: Josh Berkus @ 2015-03-31 19:52 UTC (permalink / raw)
To: pgsql-performance
All,
I currently have access to a matched pair of 20-core, 128GB RAM servers
with SSD-PCI storage, for about 2 weeks before they go into production.
Are there any performance tests people would like to see me run on
these? Otherwise, I'll just do some pgbench and DVDStore.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
@ 2015-03-31 20:41 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-01 09:02 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
0 siblings, 2 replies; 21+ messages in thread
From: Mel Llaguno @ 2015-03-31 20:41 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; pgsql-performance
It would be interesting to get raw performance benchmarks in addition to
PG specific benchmarks. I’ve been measuring raw I/O performance of a few
of our systems and run the following tests as well:
1. 10 runs of bonnie++
2. 5 runs of hdparm -Tt
3. Using a temp file created on the SSD, dd if=tempfile of=/dev/null bs=1M
count=1024 && echo 3 > /proc/sys/vm/drop_caches; dd
if=tempfileof=/dev/null bs=1M count=1024
4. Using phoronix benchmarks -> stream / ramspeed / compress-7zip
I was curious to measure the magnitude of difference between HDD -> SSD. I
would expect significant differences between SSD -> PCI-E Flash.
I’ve included some metrics from some previous runs vs. different types of
SSDs (OWC Mercury Extreme 6G which is our standard SSD, an Intel S3700
SSD, a Samsung SSD 840 PRO) vs. some standard HDD from Western Digital
and HGST. I put in a req for a 960Gb Mercury Excelsior PCI-E SSD which
hasn’t yet materialized ...
Thanks, M.
Mel Llaguno • Staff Engineer – Team Lead
Office: +1.403.264.9717 x310
www.coverity.com <http://www.coverity.com/; • Twitter: @coverity
Coverity by Synopsys
On 3/31/15, 1:52 PM, "Josh Berkus" <josh@agliodbs.com> wrote:
>All,
>
>I currently have access to a matched pair of 20-core, 128GB RAM servers
>with SSD-PCI storage, for about 2 weeks before they go into production.
> Are there any performance tests people would like to see me run on
>these? Otherwise, I'll just do some pgbench and DVDStore.
>
>--
>Josh Berkus
>PostgreSQL Experts Inc.
>http://pgexperts.com
>
>
>--
>Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-performance
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Attachments:
System Benchmarks.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet, 64.6K) download
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
@ 2015-04-01 09:02 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-03 17:44 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
1 sibling, 1 reply; 21+ messages in thread
From: Przemysław Deć @ 2015-04-01 09:02 UTC (permalink / raw)
To: Mel Llaguno <mllaguno@coverity.com>; +Cc: Josh Berkus <josh@agliodbs.com>; pgsql-performance
Maybe you will find time to benchamark xfs vs ext4 (with and without
journaling enabled on ext4).
Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X vs
RHEL 7.0 and kernel 3.10.
I was looking for some guidance what to choose and there is very poor
information about such things.
--
Przemysław Deć
Senior Solutions Architect
Linux Polska Sp. z o.o
2015-04-01 10:37 GMT+02:00 Przemysław Deć <przemyslaw.dec@linuxpolska.pl>:
> Maybe you will find time to benchamark xfs vs ext4 (with and without
> journaling enabled on ext4).
>
> Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X vs
> RHEL 7.0 and kernel 3.10.
>
> I was looking for some guidance what to choose and there is very poor
> information about such things.
>
> --
> Przemysław Deć
> Senior Solutions Architect
> Linux Polska Sp. z o.o
>
>
> 2015-03-31 22:41 GMT+02:00 Mel Llaguno <mllaguno@coverity.com>:
>
>> It would be interesting to get raw performance benchmarks in addition to
>> PG specific benchmarks. I’ve been measuring raw I/O performance of a few
>> of our systems and run the following tests as well:
>>
>> 1. 10 runs of bonnie++
>> 2. 5 runs of hdparm -Tt
>> 3. Using a temp file created on the SSD, dd if=tempfile of=/dev/null bs=1M
>> count=1024 && echo 3 > /proc/sys/vm/drop_caches; dd
>> if=tempfileof=/dev/null bs=1M count=1024
>> 4. Using phoronix benchmarks -> stream / ramspeed / compress-7zip
>>
>>
>> I was curious to measure the magnitude of difference between HDD -> SSD. I
>> would expect significant differences between SSD -> PCI-E Flash.
>>
>> I’ve included some metrics from some previous runs vs. different types of
>> SSDs (OWC Mercury Extreme 6G which is our standard SSD, an Intel S3700
>> SSD, a Samsung SSD 840 PRO) vs. some standard HDD from Western Digital
>> and HGST. I put in a req for a 960Gb Mercury Excelsior PCI-E SSD which
>> hasn’t yet materialized ...
>>
>> Thanks, M.
>>
>> Mel Llaguno • Staff Engineer – Team Lead
>> Office: +1.403.264.9717 x310
>> www.coverity.com <http://www.coverity.com/; • Twitter: @coverity
>> Coverity by Synopsys
>>
>>
>> On 3/31/15, 1:52 PM, "Josh Berkus" <josh@agliodbs.com> wrote:
>>
>> >All,
>> >
>> >I currently have access to a matched pair of 20-core, 128GB RAM servers
>> >with SSD-PCI storage, for about 2 weeks before they go into production.
>> > Are there any performance tests people would like to see me run on
>> >these? Otherwise, I'll just do some pgbench and DVDStore.
>> >
>> >--
>> >Josh Berkus
>> >PostgreSQL Experts Inc.
>> >http://pgexperts.com
>> >
>> >
>> >--
>> >Sent via pgsql-performance mailing list (
>> pgsql-performance@postgresql.org)
>> >To make changes to your subscription:
>> >http://www.postgresql.org/mailpref/pgsql-performance
>>
>>
>>
>> --
>> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org
>> )
>> To make changes to your subscription:
>> http://www.postgresql.org/mailpref/pgsql-performance
>>
>>
>
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-01 09:02 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
@ 2015-04-03 17:44 ` Mel Llaguno <mllaguno@coverity.com>
0 siblings, 0 replies; 21+ messages in thread
From: Mel Llaguno @ 2015-04-03 17:44 UTC (permalink / raw)
To: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: Josh Berkus <josh@agliodbs.com>; pgsql-performance
Although the results only focus on SATA2 HDD, these may be useful for a comparison of ext4 vs. xfs : http://www.fuzzy.cz/bench/compare-pgbench.php?type[]=btrfs-datacow-barrier:1&type[]=btrfs-datacow-nobarrier:1&type[]=btrfs-nodatacow-barrier:1&type[]=btrfs-nodatacow-nobarrier:1&type[]=ext4-writeback-barrier:1&type[]=ext4-writeback-nobarrier:1&type[]=xfs-barrier:1&type[]=xfs-nobarrier:1?
M.
________________________________
From: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
Sent: Wednesday, April 01, 2015 3:02 AM
To: Mel Llaguno
Cc: Josh Berkus; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Some performance testing?
Maybe you will find time to benchamark xfs vs ext4 (with and without journaling enabled on ext4).
Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X vs RHEL 7.0 and kernel 3.10.
I was looking for some guidance what to choose and there is very poor information about such things.
[https://ssl.gstatic.com/ui/v1/icons/mail/images/cleardot.gif]
--
Przemysław Deć
Senior Solutions Architect
Linux Polska Sp. z o.o
2015-04-01 10:37 GMT+02:00 Przemysław Deć <przemyslaw.dec@linuxpolska.pl<mailto:przemyslaw.dec@linuxpolska.pl>>:
Maybe you will find time to benchamark xfs vs ext4 (with and without journaling enabled on ext4).
Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X vs RHEL 7.0 and kernel 3.10.
I was looking for some guidance what to choose and there is very poor information about such things.
--
Przemysław Deć
Senior Solutions Architect
Linux Polska Sp. z o.o
2015-03-31 22:41 GMT+02:00 Mel Llaguno <mllaguno@coverity.com<mailto:mllaguno@coverity.com>>:
It would be interesting to get raw performance benchmarks in addition to
PG specific benchmarks. I've been measuring raw I/O performance of a few
of our systems and run the following tests as well:
1. 10 runs of bonnie++
2. 5 runs of hdparm -Tt
3. Using a temp file created on the SSD, dd if=tempfile of=/dev/null bs=1M
count=1024 && echo 3 > /proc/sys/vm/drop_caches; dd
if=tempfileof=/dev/null bs=1M count=1024
4. Using phoronix benchmarks -> stream / ramspeed / compress-7zip
I was curious to measure the magnitude of difference between HDD -> SSD. I
would expect significant differences between SSD -> PCI-E Flash.
I've included some metrics from some previous runs vs. different types of
SSDs (OWC Mercury Extreme 6G which is our standard SSD, an Intel S3700
SSD, a Samsung SSD 840 PRO) vs. some standard HDD from Western Digital
and HGST. I put in a req for a 960Gb Mercury Excelsior PCI-E SSD which
hasn't yet materialized ...
Thanks, M.
Mel Llaguno * Staff Engineer - Team Lead
Office: +1.403.264.9717 x310
www.coverity.com<http://www.coverity.com; <http://www.coverity.com/; * Twitter: @coverity
Coverity by Synopsys
On 3/31/15, 1:52 PM, "Josh Berkus" <josh@agliodbs.com<mailto:josh@agliodbs.com>> wrote:
>All,
>
>I currently have access to a matched pair of 20-core, 128GB RAM servers
>with SSD-PCI storage, for about 2 weeks before they go into production.
> Are there any performance tests people would like to see me run on
>these? Otherwise, I'll just do some pgbench and DVDStore.
>
>--
>Josh Berkus
>PostgreSQL Experts Inc.
>http://pgexperts.com
>
>
>--
>Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org<mailto:pgsql-performance@postgresql.org>)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-performance
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org<mailto:pgsql-performance@postgresql.org>)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
@ 2015-04-06 20:51 ` Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
1 sibling, 1 reply; 21+ messages in thread
From: Josh Berkus @ 2015-04-06 20:51 UTC (permalink / raw)
To: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; Mel Llaguno <mllaguno@coverity.com>; +Cc: pgsql-performance
On 04/01/2015 01:37 AM, Przemysław Deć wrote:
> Maybe you will find time to benchamark xfs vs ext4 (with and without
> journaling enabled on ext4).
>
> Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X
> vs RHEL 7.0 and kernel 3.10.
Due to how these are hosted, I can't swap out kernels.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
@ 2015-04-07 16:46 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 21+ messages in thread
From: Mel Llaguno @ 2015-04-07 16:46 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
FYI - all my tests were conducted using Ubuntu 12.04 x64 LTS (which I
believe are all 3.xx series kernels).
Mel Llaguno • Staff Engineer – Team Lead
Office: +1.403.264.9717 x310
www.coverity.com <http://www.coverity.com/; • Twitter: @coverity
Coverity by Synopsys
On 4/6/15, 2:51 PM, "Josh Berkus" <josh@agliodbs.com> wrote:
>On 04/01/2015 01:37 AM, Przemysław Deć wrote:
>> Maybe you will find time to benchamark xfs vs ext4 (with and without
>> journaling enabled on ext4).
>>
>> Nice comparison also could be rhel 6.5 with its newest kernel 2.6.32-X
>> vs RHEL 7.0 and kernel 3.10.
>
>Due to how these are hosted, I can't swap out kernels.
>
>--
>Josh Berkus
>PostgreSQL Experts Inc.
>http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
@ 2015-04-07 17:41 ` Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
0 siblings, 1 reply; 21+ messages in thread
From: Josh Berkus @ 2015-04-07 17:41 UTC (permalink / raw)
To: Mel Llaguno <mllaguno@coverity.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
On 04/07/2015 09:46 AM, Mel Llaguno wrote:
> FYI - all my tests were conducted using Ubuntu 12.04 x64 LTS (which I
> believe are all 3.xx series kernels).
If it's 3.2 or 3.5, then your tests aren't useful, I'm afraid. Both of
those kernels have known, severe, memory management issues.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
@ 2015-04-07 18:07 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
0 siblings, 1 reply; 21+ messages in thread
From: Mel Llaguno @ 2015-04-07 18:07 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
Care to elaborate? We usually do not recommend specific kernel versions
for our customers (who run on a variety of distributions). Thanks, M.
Mel Llaguno • Staff Engineer – Team Lead
Office: +1.403.264.9717 x310
www.coverity.com <http://www.coverity.com/; • Twitter: @coverity
Coverity by Synopsys
On 4/7/15, 11:41 AM, "Josh Berkus" <josh@agliodbs.com> wrote:
>On 04/07/2015 09:46 AM, Mel Llaguno wrote:
>> FYI - all my tests were conducted using Ubuntu 12.04 x64 LTS (which I
>> believe are all 3.xx series kernels).
>
>If it's 3.2 or 3.5, then your tests aren't useful, I'm afraid. Both of
>those kernels have known, severe, memory management issues.
>
>--
>Josh Berkus
>PostgreSQL Experts Inc.
>http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
@ 2015-04-08 19:05 ` Josh Berkus <josh@agliodbs.com>
2015-04-08 21:29 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-08 22:09 ` Re: Some performance testing? Michael Nolan <htfoot@gmail.com>
0 siblings, 2 replies; 21+ messages in thread
From: Josh Berkus @ 2015-04-08 19:05 UTC (permalink / raw)
To: Mel Llaguno <mllaguno@coverity.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
On 04/07/2015 11:07 AM, Mel Llaguno wrote:
> Care to elaborate? We usually do not recommend specific kernel versions
> for our customers (who run on a variety of distributions). Thanks, M.
You should.
http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html
Performance is literally 2X to 5X different between kernels.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
@ 2015-04-08 21:29 ` Mel Llaguno <mllaguno@coverity.com>
1 sibling, 0 replies; 21+ messages in thread
From: Mel Llaguno @ 2015-04-08 21:29 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
Cool. Good to know. I'll see if I can replicate these results in my environment. Thanks, M.
________________________________________
From: Josh Berkus <josh@agliodbs.com>
Sent: Wednesday, April 08, 2015 1:05 PM
To: Mel Llaguno; Przemysław Deć
Cc: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Some performance testing?
On 04/07/2015 11:07 AM, Mel Llaguno wrote:
> Care to elaborate? We usually do not recommend specific kernel versions
> for our customers (who run on a variety of distributions). Thanks, M.
You should.
http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html
Performance is literally 2X to 5X different between kernels.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
@ 2015-04-08 22:09 ` Michael Nolan <htfoot@gmail.com>
2015-04-09 15:20 ` Re: Some performance testing? Wes Vaske (wvaske) <wvaske@micron.com>
1 sibling, 1 reply; 21+ messages in thread
From: Michael Nolan @ 2015-04-08 22:09 UTC (permalink / raw)
To: Josh Berkus <josh@agliodbs.com>; +Cc: Mel Llaguno <mllaguno@coverity.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; pgsql-performance
On Wed, Apr 8, 2015 at 3:05 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 04/07/2015 11:07 AM, Mel Llaguno wrote:
> > Care to elaborate? We usually do not recommend specific kernel versions
> > for our customers (who run on a variety of distributions). Thanks, M.
>
> You should.
>
>
> http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html
>
> Performance is literally 2X to 5X different between kernels.
>
>
Josh, there seems to be an inconsistency in your blog. You say 3.10.X is
safe, but the graph you show with the poor performance seems to be from
3.13.X which as I understand it is a later kernel. Can you clarify which
3.X kernels are good to use and which are not?
--
Mike Nolan
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-08 22:09 ` Re: Some performance testing? Michael Nolan <htfoot@gmail.com>
@ 2015-04-09 15:20 ` Wes Vaske (wvaske) <wvaske@micron.com>
2015-04-10 13:21 ` Re: Some performance testing? Michael Nolan <htfoot@gmail.com>
0 siblings, 1 reply; 21+ messages in thread
From: Wes Vaske (wvaske) @ 2015-04-09 15:20 UTC (permalink / raw)
To: Michael Nolan <htfoot@gmail.com>; Josh Berkus <josh@agliodbs.com>; +Cc: Mel Llaguno <mllaguno@coverity.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; pgsql-performance
Hey Mike,
What those graphs are showing is that the new kernel reduces the IO required for the same DB load. At least, that’s how we’re supposed to interpret it.
I’d be curious to see a measure of the database load for both of those so we can verify that the new kernel does in fact provide better performance.
-Wes
From: pgsql-performance-owner@postgresql.org [mailto:pgsql-performance-owner@postgresql.org] On Behalf Of Michael Nolan
Sent: Wednesday, April 08, 2015 5:09 PM
To: Josh Berkus
Cc: Mel Llaguno; Przemysław Deć; pgsql-performance@postgresql.org
Subject: Re: [PERFORM] Some performance testing?
On Wed, Apr 8, 2015 at 3:05 PM, Josh Berkus <josh@agliodbs.com<mailto:josh@agliodbs.com>> wrote:
On 04/07/2015 11:07 AM, Mel Llaguno wrote:
> Care to elaborate? We usually do not recommend specific kernel versions
> for our customers (who run on a variety of distributions). Thanks, M.
You should.
http://www.databasesoup.com/2014/09/why-you-need-to-avoid-linux-kernel-32.html
Performance is literally 2X to 5X different between kernels.
Josh, there seems to be an inconsistency in your blog. You say 3.10.X is safe, but the graph you show with the poor performance seems to be from 3.13.X which as I understand it is a later kernel. Can you clarify which 3.X kernels are good to use and which are not?
--
Mike Nolan
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Re: Some performance testing? Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-04-08 22:09 ` Re: Some performance testing? Michael Nolan <htfoot@gmail.com>
2015-04-09 15:20 ` Re: Some performance testing? Wes Vaske (wvaske) <wvaske@micron.com>
@ 2015-04-10 13:21 ` Michael Nolan <htfoot@gmail.com>
0 siblings, 0 replies; 21+ messages in thread
From: Michael Nolan @ 2015-04-10 13:21 UTC (permalink / raw)
To: Wes Vaske (wvaske) <wvaske@micron.com>; +Cc: Josh Berkus <josh@agliodbs.com>; Mel Llaguno <mllaguno@coverity.com>; Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; pgsql-performance
On Thu, Apr 9, 2015 at 11:20 AM, Wes Vaske (wvaske) <wvaske@micron.com>
wrote:
> Hey Mike,
>
>
>
> What those graphs are showing is that the new kernel reduces the IO
> required for the same DB load. At least, that’s how we’re supposed to
> interpret it.
>
>
>
> I’d be curious to see a measure of the database load for both of those so
> we can verify that the new kernel does in fact provide better performance.
>
>
>
> *-Wes *
>
>
>
I think you're correct that I mis-interpreted the graph. I hope I was the
only one to do so.
--
Mike Nolan
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
@ 2015-04-09 09:04 Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
0 siblings, 1 reply; 21+ messages in thread
From: Graeme B. Bell @ 2015-04-09 09:04 UTC (permalink / raw)
To: pgsql-performance
>
> Josh, there seems to be an inconsistency in your blog. You say 3.10.X is
> safe, but the graph you show with the poor performance seems to be from
> 3.13.X which as I understand it is a later kernel. Can you clarify which
> 3.X kernels are good to use and which are not?
Sorry to cut in -
So far we've found kernel 3.18 to be excellent for postgres 9.3 performance (pgbench + our own queries run much faster than with the 2.6.32-504 centos 6 kernel, and we haven't encountered random stalls or slowness).
We use elrepo to get prebuilt rpms of the latest mainline stable kernel (kernel-ml).
http://elrepo.org/tiki/kernel-ml
Graeme Bell
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
@ 2015-04-09 10:39 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
0 siblings, 1 reply; 21+ messages in thread
From: Przemysław Deć @ 2015-04-09 10:39 UTC (permalink / raw)
To: Graeme B. Bell <grb@skogoglandskap.no>; +Cc: pgsql-performance
Can you say how much faster it was?
Przemek Deć
2015-04-09 11:04 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
> >
> > Josh, there seems to be an inconsistency in your blog. You say 3.10.X is
> > safe, but the graph you show with the poor performance seems to be from
> > 3.13.X which as I understand it is a later kernel. Can you clarify which
> > 3.X kernels are good to use and which are not?
>
> Sorry to cut in -
>
> So far we've found kernel 3.18 to be excellent for postgres 9.3
> performance (pgbench + our own queries run much faster than with the
> 2.6.32-504 centos 6 kernel, and we haven't encountered random stalls or
> slowness).
>
> We use elrepo to get prebuilt rpms of the latest mainline stable kernel
> (kernel-ml).
>
> http://elrepo.org/tiki/kernel-ml
>
> Graeme Bell
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
@ 2015-04-09 11:01 ` Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
0 siblings, 1 reply; 21+ messages in thread
From: Graeme B. Bell @ 2015-04-09 11:01 UTC (permalink / raw)
To: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
From a measurement I took back when we did the upgrade:
performance with 2.6: (pgbench, size 100, 32 clients)
48 651 transactions per second (read only)
6 504 transactions per second (read-write)
performance with 3.18 (pgbench, size 100, 32 clients)
129 303 transactions per second (read only)
16 895 transactions (read-write)
So that looks like 2.6x improvement to reads and writes. That was an 8 core xeon server with H710P and 4x crucial M550 SSDs in RAID, pg9.3.
Graeme Bell
On 09 Apr 2015, at 12:39, Przemysław Deć <przemyslaw.dec@linuxpolska.pl> wrote:
> Can you say how much faster it was?
>
> Przemek Deć
>
> 2015-04-09 11:04 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
> >
> > Josh, there seems to be an inconsistency in your blog. You say 3.10.X is
> > safe, but the graph you show with the poor performance seems to be from
> > 3.13.X which as I understand it is a later kernel. Can you clarify which
> > 3.X kernels are good to use and which are not?
>
> Sorry to cut in -
>
> So far we've found kernel 3.18 to be excellent for postgres 9.3 performance (pgbench + our own queries run much faster than with the 2.6.32-504 centos 6 kernel, and we haven't encountered random stalls or slowness).
>
> We use elrepo to get prebuilt rpms of the latest mainline stable kernel (kernel-ml).
>
> http://elrepo.org/tiki/kernel-ml
>
> Graeme Bell
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
@ 2015-04-09 11:56 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 13:35 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
0 siblings, 1 reply; 21+ messages in thread
From: Przemysław Deć @ 2015-04-09 11:56 UTC (permalink / raw)
To: Graeme B. Bell <grb@skogoglandskap.no>; +Cc: pgsql-performance
Wow, thats huge performance gain.
And it was on ext4?
--
Linux Polska Sp. z o.o.
Przemysław Deć - Senior Solutions Architect
RHCSA, RHCJA, PostgreSQL Professional Certification
mob: +48 519 130 141
email: pd@linuxpolska.pl
www.linuxpolska.pl
___________________________________________________________________________________________________________________________
Linux Polska Sp. z o. o. Al. Jerozolimskie 123A (26 p.); 02-017 Warszawa;
tel. (+48) 222139571; fax (+48)222139671
KRS - 0000326158 Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział
Gospodarczy KRS
Kapitał zakładowy wpłacony 1 000 500PLN; NIP 7010181018; REGON 141791601
[image: Open Source Day 2015] <http://opensourceday.pl/;
2015-04-09 13:01 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
>
> From a measurement I took back when we did the upgrade:
>
> performance with 2.6: (pgbench, size 100, 32 clients)
>
> 48 651 transactions per second (read only)
> 6 504 transactions per second (read-write)
>
>
> performance with 3.18 (pgbench, size 100, 32 clients)
>
> 129 303 transactions per second (read only)
> 16 895 transactions (read-write)
>
>
> So that looks like 2.6x improvement to reads and writes. That was an 8
> core xeon server with H710P and 4x crucial M550 SSDs in RAID, pg9.3.
>
> Graeme Bell
>
>
>
>
>
> On 09 Apr 2015, at 12:39, Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
> wrote:
>
> > Can you say how much faster it was?
> >
> > Przemek Deć
> >
> > 2015-04-09 11:04 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
> > >
> > > Josh, there seems to be an inconsistency in your blog. You say 3.10.X
> is
> > > safe, but the graph you show with the poor performance seems to be from
> > > 3.13.X which as I understand it is a later kernel. Can you clarify
> which
> > > 3.X kernels are good to use and which are not?
> >
> > Sorry to cut in -
> >
> > So far we've found kernel 3.18 to be excellent for postgres 9.3
> performance (pgbench + our own queries run much faster than with the
> 2.6.32-504 centos 6 kernel, and we haven't encountered random stalls or
> slowness).
> >
> > We use elrepo to get prebuilt rpms of the latest mainline stable kernel
> (kernel-ml).
> >
> > http://elrepo.org/tiki/kernel-ml
> >
> > Graeme Bell
> >
> > --
> > Sent via pgsql-performance mailing list (
> pgsql-performance@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-performance
> >
>
>
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
@ 2015-04-09 13:35 ` Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 15:15 ` Re: Some performance testing? Scott Marlowe <scott.marlowe@gmail.com>
0 siblings, 1 reply; 21+ messages in thread
From: Graeme B. Bell @ 2015-04-09 13:35 UTC (permalink / raw)
To: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; +Cc: pgsql-performance
ext4 settings
ext4, nobarrier
noatime+nodatime,
stripe&stride aligned between raid10 & ext4 correctly.
Some other useful things to know
-- h710p
readahead disabled on H710P
writeback cache enabled on H710P
Direct IO enabled on H710P
-- os filesystem settings
linux readahead enabled (16384),
nr_requests=975
NOOP scheduler
non-NUMA
-- pg
io_concurrency on
async commit.*** see below!
All settings were kept identical on the server before and after the kernel change, so this performance increase can be entirely attributed to the newer kernel and its synergies with our configuration. 3.18 contains about 5-10 years of linux kernel development vs. 2.6 kernels (except where backported).
I have conducted quite a lot of plug-pull testing with diskchecker.pl, and rather a lot of testing of scheduling/IO/RAID controller/etc parameters. The OS/RAID controller/file system settings are as fast as I've been able to achieve without compromising database integrity (please note: this server can run async_commit because of the work we use it for, but we do not use that setting on our other main production servers).
Our local DBs run extremely nicely for all our normal queries which involve quite a mix of random small IO and full-table operations on e.g. 20GB+ tables , so they're not optimised for pgbench specifically.
Graeme Bell
On 09 Apr 2015, at 13:56, Przemysław Deć <przemyslaw.dec@linuxpolska.pl> wrote:
> Wow, thats huge performance gain.
> And it was on ext4?
>
> --
> Linux Polska Sp. z o.o.
> Przemysław Deć - Senior Solutions Architect
> RHCSA, RHCJA, PostgreSQL Professional Certification
> mob: +48 519 130 141
> email: pd@linuxpolska.pl
> www.linuxpolska.pl
> ___________________________________________________________________________________________________________________________
> Linux Polska Sp. z o. o. Al. Jerozolimskie 123A (26 p.); 02-017 Warszawa; tel. (+48) 222139571; fax (+48)222139671
> KRS - 0000326158 Sąd Rejonowy dla M. St. Warszawy w Warszawie, XII Wydział Gospodarczy KRS
> Kapitał zakładowy wpłacony 1 000 500PLN; NIP 7010181018; REGON 141791601
> <Mail Attachment.jpeg>
>
>
> 2015-04-09 13:01 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
>
> From a measurement I took back when we did the upgrade:
>
> performance with 2.6: (pgbench, size 100, 32 clients)
>
> 48 651 transactions per second (read only)
> 6 504 transactions per second (read-write)
>
>
> performance with 3.18 (pgbench, size 100, 32 clients)
>
> 129 303 transactions per second (read only)
> 16 895 transactions (read-write)
>
>
> So that looks like 2.6x improvement to reads and writes. That was an 8 core xeon server with H710P and 4x crucial M550 SSDs in RAID, pg9.3.
>
> Graeme Bell
>
>
>
>
>
> On 09 Apr 2015, at 12:39, Przemysław Deć <przemyslaw.dec@linuxpolska.pl> wrote:
>
> > Can you say how much faster it was?
> >
> > Przemek Deć
> >
> > 2015-04-09 11:04 GMT+02:00 Graeme B. Bell <grb@skogoglandskap.no>:
> > >
> > > Josh, there seems to be an inconsistency in your blog. You say 3.10.X is
> > > safe, but the graph you show with the poor performance seems to be from
> > > 3.13.X which as I understand it is a later kernel. Can you clarify which
> > > 3.X kernels are good to use and which are not?
> >
> > Sorry to cut in -
> >
> > So far we've found kernel 3.18 to be excellent for postgres 9.3 performance (pgbench + our own queries run much faster than with the 2.6.32-504 centos 6 kernel, and we haven't encountered random stalls or slowness).
> >
> > We use elrepo to get prebuilt rpms of the latest mainline stable kernel (kernel-ml).
> >
> > http://elrepo.org/tiki/kernel-ml
> >
> > Graeme Bell
> >
> > --
> > Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> > To make changes to your subscription:
> > http://www.postgresql.org/mailpref/pgsql-performance
> >
>
>
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 13:35 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
@ 2015-04-09 15:15 ` Scott Marlowe <scott.marlowe@gmail.com>
2015-04-10 13:29 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-14 19:58 ` Re: Some performance testing? Josh Berkus <josh@agliodbs.com>
0 siblings, 2 replies; 21+ messages in thread
From: Scott Marlowe @ 2015-04-09 15:15 UTC (permalink / raw)
To: Graeme B. Bell <grb@skogoglandskap.no>; +Cc: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; pgsql-performance
On Thu, Apr 9, 2015 at 7:35 AM, Graeme B. Bell <grb@skogoglandskap.no> wrote:
> ext4 settings
>
> ext4, nobarrier
> noatime+nodatime,
> stripe&stride aligned between raid10 & ext4 correctly.
>
>
> Some other useful things to know
>
> -- h710p
> readahead disabled on H710P
> writeback cache enabled on H710P
> Direct IO enabled on H710P
>
> -- os filesystem settings
> linux readahead enabled (16384),
> nr_requests=975
> NOOP scheduler
> non-NUMA
>
> -- pg
> io_concurrency on
> async commit.*** see below!
>
> All settings were kept identical on the server before and after the kernel change, so this performance increase can be entirely attributed to the newer kernel and its synergies with our configuration. 3.18 contains about 5-10 years of linux kernel development vs. 2.6 kernels (except where backported).
>
> I have conducted quite a lot of plug-pull testing with diskchecker.pl, and rather a lot of testing of scheduling/IO/RAID controller/etc parameters. The OS/RAID controller/file system settings are as fast as I've been able to achieve without compromising database integrity (please note: this server can run async_commit because of the work we use it for, but we do not use that setting on our other main production servers).
>
> Our local DBs run extremely nicely for all our normal queries which involve quite a mix of random small IO and full-table operations on e.g. 20GB+ tables , so they're not optimised for pgbench specifically.
It would be handy to see a chart comparing 3.11, 3.13 and 3.18 as
well, to see if most / any of those performance gains came in earlier
kernels, but after 2.6 or 3.2 etc.
Can confirm that for pg purposes, 3.2 is basically broken in some not
to great ways. We've had servers that were overloaded at load factors
of 20 or 30 drop down to 5 or less with just a change from 3.2 to
3.11/3.13 on ubuntu 12.04
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 13:35 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 15:15 ` Re: Some performance testing? Scott Marlowe <scott.marlowe@gmail.com>
@ 2015-04-10 13:29 ` Graeme B. Bell <grb@skogoglandskap.no>
1 sibling, 0 replies; 21+ messages in thread
From: Graeme B. Bell @ 2015-04-10 13:29 UTC (permalink / raw)
To: Scott Marlowe <scott.marlowe@gmail.com>; +Cc: Przemysław Deć <przemyslaw.dec@linuxpolska.pl>; pgsql-performance
The 3.10 mainline had a bug which crashes on boot for us, I think it was the network card driver for the R620. Can't recall.
The equipment is in continual use so unfortunately we can't test other kernels at the moment but hopefully someone else may be interested.
It's probably valuable to compare against 3.19 too for anyone who doesn't want to delve into kernel archeology.
Kernel performance gains will be sensitive to things like VM settings, scheduler settings, NUMA, hardware choice, BIOS settings etc, use of virtualisation, so it depends which codepaths you end up running. So it may not be meaningful to compare kernels without a range of system configurations to measure against. ie. Your mileage will probably vary depending on how far you've tuned your disk configuration, memory, etc.
Graeme Bell
On 09 Apr 2015, at 17:15, Scott Marlowe <scott.marlowe@gmail.com> wrote:
> On Thu, Apr 9, 2015 at 7:35 AM, Graeme B. Bell <grb@skogoglandskap.no> wrote:
>> ext4 settings
>>
>> ext4, nobarrier
>> noatime+nodatime,
>> stripe&stride aligned between raid10 & ext4 correctly.
>>
>>
>> Some other useful things to know
>>
>> -- h710p
>> readahead disabled on H710P
>> writeback cache enabled on H710P
>> Direct IO enabled on H710P
>>
>> -- os filesystem settings
>> linux readahead enabled (16384),
>> nr_requests=975
>> NOOP scheduler
>> non-NUMA
>>
>> -- pg
>> io_concurrency on
>> async commit.*** see below!
>>
>> All settings were kept identical on the server before and after the kernel change, so this performance increase can be entirely attributed to the newer kernel and its synergies with our configuration. 3.18 contains about 5-10 years of linux kernel development vs. 2.6 kernels (except where backported).
>>
>> I have conducted quite a lot of plug-pull testing with diskchecker.pl, and rather a lot of testing of scheduling/IO/RAID controller/etc parameters. The OS/RAID controller/file system settings are as fast as I've been able to achieve without compromising database integrity (please note: this server can run async_commit because of the work we use it for, but we do not use that setting on our other main production servers).
>>
>> Our local DBs run extremely nicely for all our normal queries which involve quite a mix of random small IO and full-table operations on e.g. 20GB+ tables , so they're not optimised for pgbench specifically.
>
> It would be handy to see a chart comparing 3.11, 3.13 and 3.18 as
> well, to see if most / any of those performance gains came in earlier
> kernels, but after 2.6 or 3.2 etc.
>
> Can confirm that for pg purposes, 3.2 is basically broken in some not
> to great ways. We've had servers that were overloaded at load factors
> of 20 or 30 drop down to 5 or less with just a change from 3.2 to
> 3.11/3.13 on ubuntu 12.04
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
* Re: Some performance testing?
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Re: Some performance testing? Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 13:35 ` Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 15:15 ` Re: Some performance testing? Scott Marlowe <scott.marlowe@gmail.com>
@ 2015-04-14 19:58 ` Josh Berkus <josh@agliodbs.com>
1 sibling, 0 replies; 21+ messages in thread
From: Josh Berkus @ 2015-04-14 19:58 UTC (permalink / raw)
To: pgsql-performance
Scott,
> Can confirm that for pg purposes, 3.2 is basically broken in some not
> to great ways. We've had servers that were overloaded at load factors
> of 20 or 30 drop down to 5 or less with just a change from 3.2 to
> 3.11/3.13 on ubuntu 12.04
That's correct, and 3.5 shares the same problems. The underlying issue
was that 3.X was tweaked to be MUCH more aggressive about
cache-clearing, to the point where it would be evicting data from the FS
cache which had just been read in and hadn't even been used yet. For
some reason, this aggressive eviction got worse the more processes on
the system which were using the FS cache, so where you really see it is
when you have more processes with cache than you have cores.
It's pretty easy to demonstrate just using pgbench, with a database
larger than RAM, and 2X as many clients as cores. You'll see that
kernels 3.2 and 3.5 will do 3X to 5X as much IO for the same workload as
3.10 and later will do.
Greame,
On 04/09/2015 04:01 AM, Graeme B. Bell wrote:> performance with 2.6:
(pgbench, size 100, 32 clients)
>
> 48 651 transactions per second (read only)
> 6 504 transactions per second (read-write)
>
>
> performance with 3.18 (pgbench, size 100, 32 clients)
>
> 129 303 transactions per second (read only)
> 16 895 transactions (read-write)
Thanks for that data! I'm glad to see that 3.18 has improved so much.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
^ permalink raw reply [nested|flat] 21+ messages in thread
end of thread, other threads:[~2015-04-14 19:58 UTC | newest]
Thread overview: 21+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2015-03-31 19:52 Some performance testing? Josh Berkus <josh@agliodbs.com>
2015-03-31 20:41 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-01 09:02 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-03 17:44 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-06 20:51 ` Josh Berkus <josh@agliodbs.com>
2015-04-07 16:46 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-07 17:41 ` Josh Berkus <josh@agliodbs.com>
2015-04-07 18:07 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-08 19:05 ` Josh Berkus <josh@agliodbs.com>
2015-04-08 21:29 ` Mel Llaguno <mllaguno@coverity.com>
2015-04-08 22:09 ` Michael Nolan <htfoot@gmail.com>
2015-04-09 15:20 ` Wes Vaske (wvaske) <wvaske@micron.com>
2015-04-10 13:21 ` Michael Nolan <htfoot@gmail.com>
2015-04-09 09:04 Re: Some performance testing? Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 10:39 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 11:01 ` Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 11:56 ` Przemysław Deć <przemyslaw.dec@linuxpolska.pl>
2015-04-09 13:35 ` Graeme B. Bell <grb@skogoglandskap.no>
2015-04-09 15:15 ` Scott Marlowe <scott.marlowe@gmail.com>
2015-04-10 13:29 ` Graeme B. Bell <grb@skogoglandskap.no>
2015-04-14 19:58 ` Josh Berkus <josh@agliodbs.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox