Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TLLV1-0000rB-PN for pgsql-performance@postgresql.org; Mon, 08 Oct 2012 22:08:31 +0000 Received: from mail-lb0-f174.google.com ([209.85.217.174]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1TLLUz-0001qj-KS for pgsql-performance@postgresql.org; Mon, 08 Oct 2012 22:08:31 +0000 Received: by mail-lb0-f174.google.com with SMTP id n3so3293216lbo.19 for ; Mon, 08 Oct 2012 15:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to:x-mailer; bh=99rURml2HsVJKx5zaYG7sqH7Z8wziVi2ILahi23qjSE=; b=0SyYdtkBI0pKkDWZXxsLP+Tq4zuPBiMKRoYK9H9pdCo0Lv+iSZnvbc66ypsb0OlwEl bmYnu8aViv302CfBQPbINsDWorGZlQWQTpz/xQaozv9l3m+eqGY+m0RAXqusCMDdWWaO OPdbnpBSKKmARN6q/zMDnL24d9qg9rmjUZvJULM74f1q2Fo3eiuYrSyEc85PYvLX08P6 pPJ9ZK3W/kXUm3/Fnl6S+TzQGaYMUtUUEVBCxB18VQxIVTen368w80rQ50sr+ZCB9Gzd 21M7wOYTVRnJQa4T7d0QXr9sqOYNj8zu8/6Wu4+o3Rag8darUQrtpQ34hn1Kri8ZjHGR qNGQ== Received: by 10.152.108.42 with SMTP id hh10mr4475467lab.4.1349734108938; Mon, 08 Oct 2012 15:08:28 -0700 (PDT) Received: from [192.168.20.2] (128-69-108-68.broadband.corbina.ru. [128.69.108.68]) by mx.google.com with ESMTPS id h8sm5807954lbi.13.2012.10.08.15.08.27 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 08 Oct 2012 15:08:28 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_B1E23F43-2B32-4783-A986-C5A70F18A4AE" Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: Two identical systems, radically different performance From: Evgeny Shishkin In-Reply-To: Date: Tue, 9 Oct 2012 02:08:29 +0400 Cc: pgsql-performance@postgresql.org Message-Id: <099BB618-4F80-4200-9564-4F06231D8D57@gmail.com> References: <4CB0E3B6-CF1E-4B54-9900-E8356744F0AA@gmail.com> To: Craig James X-Mailer: Apple Mail (2.1498) X-Pg-Spam-Score: -2.7 (--) X-Archive-Number: 201210/60 X-Sequence-Number: 48019 --Apple-Mail=_B1E23F43-2B32-4783-A986-C5A70F18A4AE Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On Oct 9, 2012, at 2:06 AM, Craig James wrote: >=20 >=20 > On Mon, Oct 8, 2012 at 2:57 PM, Evgeny Shishkin = wrote: >=20 > On Oct 9, 2012, at 1:45 AM, Craig James wrote: >=20 >> I tested both the RAID10 data disk and the RAID1 xlog disk with = bonnie++. The xlog disks were almost identical in performance. The = RAID10 pg-data disks looked like this: >>=20 >> Old server: >> Version 1.96 ------Sequential Output------ --Sequential Input- = --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- = --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = /sec %CP >> xenon 24064M 687 99 203098 26 81904 16 3889 96 403747 = 31 737.6 31 >> Latency 20512us 469ms 394ms 21402us 396ms = 112ms >> Version 1.96 ------Sequential Create------ --------Random = Create-------- >> xenon -Create-- --Read--- -Delete-- -Create-- --Read--- = -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = /sec %CP >> 16 15953 27 +++++ +++ +++++ +++ +++++ +++ +++++ +++ = +++++ +++ >> Latency 43291us 857us 519us 1588us 37us = 178us >> = 1.96,1.96,xenon,1,1349726125,24064M,,687,99,203098,26,81904,16,3889,96,403= 747,31,737.6,31,16,,,,,15953,27,+++++,+++,+++++,++\ >> = +,+++++,+++,+++++,+++,+++++,+++,20512us,469ms,394ms,21402us,396ms,112ms,43= 291us,857us,519us,1588us,37us,178us >>=20 >>=20 >> New server: >> Version 1.96 ------Sequential Output------ --Sequential Input- = --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- = --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = /sec %CP >> zinc 24064M 862 99 212143 54 96008 14 4921 99 279239 = 17 752.0 23 >> Latency 15613us 598ms 597ms 2764us 398ms = 215ms >> Version 1.96 ------Sequential Create------ --------Random = Create-------- >> zinc -Create-- --Read--- -Delete-- -Create-- --Read--- = -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = /sec %CP >> 16 20380 26 +++++ +++ +++++ +++ +++++ +++ +++++ +++ = +++++ +++ >> Latency 487us 627us 407us 972us 29us = 262us >> = 1.96,1.96,zinc,1,1349722017,24064M,,862,99,212143,54,96008,14,4921,99,2792= 39,17,752.0,23,16,,,,,20380,26,+++++,+++,+++++,+++\ >> = ,+++++,+++,+++++,+++,+++++,+++,15613us,598ms,597ms,2764us,398ms,215ms,487u= s,627us,407us,972us,29us,262us >=20 > Sequential Input on the new one is 279MB/s, on the old 400MB/s.=20 >=20 >=20 > But why? What have I overlooked? blockdev --setra 32000 ? Also you benchmarked volume for pgdata? Can you provide benchmarks for = wal volume? >=20 > Thanks, > Craig >=20 --Apple-Mail=_B1E23F43-2B32-4783-A986-C5A70F18A4AE Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=iso-8859-1
On Oct 9, 2012, at 2:06 AM, Craig James <cjames@emolecules.com> wrote:



On Mon, Oct 8, 2012 at 2:57 PM, Evgeny Shishkin <itparanoia@gmail.com> wrote:

On Oct 9, 2012, at 1:45 AM, Craig James <cjames@emolecules.com> wrote:

I tested both the RAID10 data disk and the RAID1 xlog disk with bonnie++.  The xlog disks were almost identical in performance.  The RAID10 pg-data disks looked like this:

Old server:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
xenon        24064M   687  99 203098  26 81904  16  3889  96 403747  31 737.6  31
Latency             20512us     469ms     394ms   21402us     396ms     112ms
Version  1.96       ------Sequential Create------ --------Random Create--------
xenon               -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 15953  27 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency             43291us     857us     519us    1588us      37us     178us
1.96,1.96,xenon,1,1349726125,24064M,,687,99,203098,26,81904,16,3889,96,403747,31,737.6,31,16,,,,,15953,27,+++++,+++,+++++,++\
+,+++++,+++,+++++,+++,+++++,+++,20512us,469ms,394ms,21402us,396ms,112ms,43291us,857us,519us,1588us,37us,178us


New server:
Version  1.96       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
zinc         24064M   862  99 212143  54 96008  14  4921  99 279239  17 752.0  23
Latency             15613us     598ms     597ms    2764us     398ms     215ms
Version  1.96       ------Sequential Create------ --------Random Create--------
zinc                -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 20380  26 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               487us     627us     407us     972us      29us     262us
1.96,1.96,zinc,1,1349722017,24064M,,862,99,212143,54,96008,14,4921,99,279239,17,752.0,23,16,,,,,20380,26,+++++,+++,+++++,+++\
,+++++,+++,+++++,+++,+++++,+++,15613us,598ms,597ms,2764us,398ms,215ms,487us,627us,407us,972us,29us,262us

Sequential Input on the new one is 279MB/s, on the old 400MB/s. 


But why? What have I overlooked?

blockdev --setra 32000 ?
Also you benchmarked volume for pgdata? Can you provide benchmarks for wal volume?

Thanks,
Craig


--Apple-Mail=_B1E23F43-2B32-4783-A986-C5A70F18A4AE--