Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1Za75s-000758-Jh for pgsql-performance@arkaria.postgresql.org; Thu, 10 Sep 2015 19:01:12 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.84) (envelope-from ) id 1Za75r-0007gY-T5 for pgsql-performance@arkaria.postgresql.org; Thu, 10 Sep 2015 19:01:11 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA384:256) (Exim 4.84) (envelope-from ) id 1Za74C-0005mo-59; Thu, 10 Sep 2015 18:59:28 +0000 Received: from smtp-outbound-1.vmware.com ([208.91.2.12]) by magus.postgresql.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84) (envelope-from ) id 1Za741-0000DI-IQ; Thu, 10 Sep 2015 18:59:26 +0000 Received: from sc9-mailhost1.vmware.com (sc9-mailhost1.vmware.com [10.113.161.71]) by smtp-outbound-1.vmware.com (Postfix) with ESMTP id 56E8E28637; Thu, 10 Sep 2015 12:01:43 -0700 (PDT) Received: from EX13-CAS-002.vmware.com (ex13-cas-002.vmware.com [10.113.191.52]) by sc9-mailhost1.vmware.com (Postfix) with ESMTP id B48D6189C3; Thu, 10 Sep 2015 11:59:12 -0700 (PDT) Received: from EX13-MBX-000.vmware.com (10.113.191.20) by EX13-MBX-021.vmware.com (10.113.191.41) with Microsoft SMTP Server (TLS) id 15.0.1076.9; Thu, 10 Sep 2015 11:59:12 -0700 Received: from EX13-MBX-000.vmware.com ([fe80::c036:65fc:73b6:ff05]) by EX13-MBX-000.vmware.com ([fe80::c036:65fc:73b6:ff05%15]) with mapi id 15.00.1076.010; Thu, 10 Sep 2015 11:59:12 -0700 From: Reza Taheri To: "pgsql-performance@postgresql.org" CC: Alvaro Herrera Subject: Announcing the public availability of the TPCx-V prototype Thread-Topic: Announcing the public availability of the TPCx-V prototype Thread-Index: AdDr+sQGaz067ieQTGmSRmol6IxOvA== Date: Thu, 10 Sep 2015 18:59:11 +0000 Message-ID: <0cea9723abc844338345e460e55be3b7@EX13-MBX-000.vmware.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [10.113.160.246] Content-Type: multipart/alternative; boundary="_000_0cea9723abc844338345e460e55be3b7EX13MBX000vmwarecom_" MIME-Version: 1.0 X-Pg-Spam-Score: -6.9 (------) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org --_000_0cea9723abc844338345e460e55be3b7EX13MBX000vmwarecom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hello PGSQL performance community, You might remember me from these past postings: * In 2012, we announced that the TPC was using PostgreSQL in a benc= hmark for virtualized databases. Unlike the traditional TPC Enterprise benc= hmarks, which are defined in paper specifications, this Express benchmark c= omes with a complete end-to-end benchmarking kit, which at this point in ti= me, runs on Linux VMs (we have tested it with PGSQL 9.3 on RHEL 7.1) on the= X86 architecture. On that occasion, I was looking for a PGSQL feature similar to MS SQL Serve= r's "clustered indexes" or Oracle's "index-only-tables". (I was told about = "index-only scans" which helped but not as much as clustered indexes or IOT= ) * In 2013, I asked about a performance hit during checkpoints, whic= h was quickly diagnosed as the well-known dirty_background_bytes problem * Last year, I asked about the high rate of transaction aborts due = to serialization failures. It turned out that even dbt5, which implements T= PC-E, was running into the same issue, and simply resubmitting the transact= ion was an acceptable solution We are done with the benchmark development, and it has entered TPC's "Publi= c Review" phase. In parallel, the TPC council has approved the public relea= se of a prototype of the benchmark kit to gather more experimental data, an= d to speed up the approval of the benchmark. The Benchmark Specification, the Benchmark Kit, and the User's Guide are av= ailable from tpc.org/tpcx-v. In addition to these components of the benchma= rk standard, the subcommittee members have developed two other tools to hel= p benchmarkers get a fast start: a downloadable VM in the form of an ovf-f= ormat VM template that contains a complete benchmarking environment with al= l the software already installed and pre-configured; and a PowerCLI script = that, in the VMware vSphere environment, allows you to quickly clone a VM t= emplate into a large number of benchmark VMs. The review period is open until Thursday, October 15th. The subcommittee in= tends to resolve the comments from the Formal Review by November 12th, and = bring forward a motion to the Council to approve the Benchmark Standard. We would love to get feedback from the PGSQL community. The subcommittee is= taking feedback via FogBugz at www.tpc.org/bugdb= under the project "TPC-Virt". Anticipating some of the questions/comments, here are some thoughts: - We wish we had a more representative workload; with multiple kit= s for multiple hardware architectures; etc. Those are questions for another= day. Having an end-to-end database benchmarking kit for X86 virtualization= with a workload derived from TPC-E is a pretty good first step. - You don't have to run the full multi-VM configuration of TPCx-V = if you are just playing with the kit and not intending to have an audited r= esult. You can run the benchmark on a single database on a single VM (or ev= en a native, un-virtualized server!) This would be very similar to a simple= TPC-E config. If we can get feedback based on a single-VM config, we will = still be grateful - Having said that, there are enough subtle differences between th= e TPC-E schema and the TPCx-V schema that running TPCx-V on a single VM doe= sn't exactly take you to TPC-E. Thanks, Reza Taheri for the TPCx-V subcommittee --_000_0cea9723abc844338345e460e55be3b7EX13MBX000vmwarecom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hello PGSQL performance community,

You might remember me from these past postings:=

·         In 2012, we announced that the TPC was using= PostgreSQL in a benchmark for virtualized databases. Unlike the traditiona= l TPC Enterprise benchmarks, which are defined in paper specifications, thi= s Express benchmark comes with a complete end-to-end benchmarking kit, which at this point in time, runs on= Linux VMs (we have tested it with PGSQL 9.3 on RHEL 7.1) on the X86 archit= ecture.
On that occasion, I was looking for a PGSQL feature similar to MS SQL Serve= r’s “clustered indexes” or Oracle’s “index-on= ly-tables”. (I was told about “index-only scans” which he= lped but not as much as clustered indexes or IOT)

·         In 2013, I asked about a performance hit dur= ing checkpoints, which was quickly diagnosed as the well-known dirty_backgr= ound_bytes problem

·         Last year, I asked about the high rate of tr= ansaction aborts due to serialization failures. It turned out that even dbt= 5, which implements TPC-E, was running into the same issue, and simply resu= bmitting the transaction was an acceptable solution

 

We are done with the benchmark development, and it h= as entered TPC’s “Public Review” phase. In parallel, the = TPC council has approved the public release of a prototype of the benchmark= kit to gather more experimental data, and to speed up the approval of the benchmark.

 

The Benchmark Specification, the Benchmark Kit, and = the User’s Guide are available from tpc.org/tpcx-v. In addition to th= ese components of the benchmark standard, the subcommittee members have dev= eloped two other tools to help benchmarkers get a fast start:  a downloadable VM in the form of an ovf-format VM = template that contains a complete benchmarking environment with all the sof= tware already installed and pre-configured; and a PowerCLI script that, in = the VMware vSphere environment, allows you to quickly clone a VM template into a large number of benchmark VMs.

 

The review period is open until Thursday, October 15= th. The subcommittee intends to resolve the comments from the Formal Review= by November 12th, and bring forward a motion to the Council to approve the= Benchmark Standard.

 

We would love to get feedback from the PGSQL communi= ty. The subcommittee is taking feedback via FogBugz at www.tpc.org/bugdb under the projec= t “TPC-Virt”.

 

Anticipating some of the questions/comments, here ar= e some thoughts:

-       = ;   We wish we had a more representative workload; with= multiple kits for multiple hardware architectures; etc. Those are question= s for another day. Having an end-to-end database benchmarking kit for X86 v= irtualization with a workload derived from TPC-E is a pretty good first step.

-       = ;   You don’t have to run the full multi-VM confi= guration of TPCx-V if you are just playing with the kit and not intending t= o have an audited result. You can run the benchmark on a single database on= a single VM (or even a native, un-virtualized server!) This would be very similar to a simple TPC-E config. If we can ge= t feedback based on a single-VM config, we will still be grateful

-     &= nbsp;    Having said that, there are enough subtle differenc= es between the TPC-E schema and the TPCx-V schema that running TPCx-V on a = single VM doesn’t exactly take you to TPC-E.

 

Thanks,

Reza Taheri for the TPCx-V subcommittee

--_000_0cea9723abc844338345e460e55be3b7EX13MBX000vmwarecom_--