Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id F396D6329BE for ; Mon, 14 Jun 2010 15:28:10 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 84274-04 for ; Mon, 14 Jun 2010 18:28:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from bay0-omc3-s8.bay0.hotmail.com (bay0-omc3-s8.bay0.hotmail.com [65.54.190.146]) by mail.postgresql.org (Postfix) with ESMTP id 0D450632876 for ; Mon, 14 Jun 2010 15:28:02 -0300 (ADT) Received: from BAY154-W57 ([65.54.190.189]) by bay0-omc3-s8.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Mon, 14 Jun 2010 11:28:01 -0700 Message-ID: Content-Type: multipart/alternative; boundary="_f4d244f6-548a-4dad-a3e6-508274af7dc1_" X-Originating-IP: [115.167.42.128] From: MUHAMMAD ASIF To: CC: Subject: =?windows-1256?Q?RE:_[PERFO?= =?windows-1256?Q?RM]_Dbt2_w?= =?windows-1256?Q?ith_postgr?= =?windows-1256?Q?es_issues_?= =?windows-1256?Q?on_CentOS-?= =?windows-1256?Q?5.3=FE?= Date: Tue, 15 Jun 2010 00:28:01 +0600 Importance: Normal In-Reply-To: References: , MIME-Version: 1.0 X-OriginalArrivalTime: 14 Jun 2010 18:28:01.0600 (UTC) FILETIME=[524C1C00:01CB0BEF] X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.791 tagged_above=-10 required=5 tests=BAYES_50=0.8, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201006/250 X-Sequence-Number: 39106 --_f4d244f6-548a-4dad-a3e6-508274af7dc1_ Content-Type: text/plain; charset="windows-1256" Content-Transfer-Encoding: 8bit I am facing sar related issues on Redhat Enterprise Linux64 5.4 too (60G Ram, No Swap space, Xeon Processor). sar -o /var/dbt2_data/PG/Output/driver/dbt2-sys1/sar_raw.out 60 204 |___ sadc 60 205 -z /var/dbt2_data/PG/Output/driver/dbt2-sys1/sar_raw.out It generates following sar data i.e. …. …. 03:52:43 AM 2.31 03:53:43 AM 2.31 03:54:43 AM 2.28 03:55:43 AM 2.31 03:56:43 AM 1.67 03:57:43 AM 0.29 03:58:43 AM 0.29 04:00:43 AM 0.30 04:04:00 AM 3.52 04:07:07 AM 0.30 04:09:36 AM 0.23 04:12:04 AM 0.36 04:14:25 AM 0.23 04:16:45 AM 0.26 04:19:10 AM 0.24 04:21:30 AM 0.38 04:23:55 AM 0.24 04:26:25 AM 0.35 04:28:48 AM 0.24 04:31:10 AM 0.27 04:33:40 AM 0.33 04:36:45 AM 0.41 04:39:12 AM 0.27 04:41:41 AM 0.26 04:44:11 AM 0.33 04:46:35 AM 0.25 04:49:06 AM 0.33 04:51:27 AM 0.27 04:53:56 AM 0.23 04:56:19 AM 0.36 04:58:43 AM 0.24 05:01:10 AM 0.35 05:03:43 AM 0.33 05:06:53 AM 0.29 05:09:25 AM 0.23 …. …. To fix this issue I have modified the sysstat-9.1.2/sadc.c and replaced signal based pause (That is not real time) with "select" based pause. That fixed the issue. Thanks. ------------------------------------------------------------------------------------------------------------------------------------------------------- sadc.c.patch ------------------------------------------------------------------------------------------------------------------------------------------------------- --- sadc.c.org 2010-06-14 21:44:18.000000000 +0500 +++ sadc.c 2010-06-14 22:52:51.693211184 +0500 @@ -33,6 +33,10 @@ #include #include +#include +#include +#include + #include "version.h" #include "sa.h" #include "rd_stats.h" @@ -792,6 +796,15 @@ } } +void pause_new( void ) +{ + struct timeval tvsel; + tvsel.tv_sec = interval; + tvsel.tv_usec = 0; + + select( 0, NULL, NULL, NULL, &tvsel ); +} + /* *************************************************************************** * Main loop: Read stats from the relevant sources and display them. @@ -899,7 +912,7 @@ } if (count) { - pause(); + pause_new(); } /* Rotate activity file if necessary */ ------------------------------------------------------------------------------------------------------------------------------------------------------- Best Regards, Asif Naeem > Date: Wed, 21 Apr 2010 18:10:35 -0700 > Subject: Re: [PERFORM] Dbt2 with postgres issues on CentOS-5.3þ > From: markwkm@gmail.com > To: anaeem.it@hotmail.com > CC: pgsql-performance@postgresql.org > > 2010/4/20 MUHAMMAD ASIF : > > Hi, > > > > I am using dbt2 on Linux 64 (CentOS release 5.3 (Final)) . I have compiled > > latest postgresql-8.4.3 code on the machine and run dbt2 against it. I am > > little confused about the results. I ran dbt2 with the following > > configuration i.e. > > > > DBT2 Options : > > WAREHOUSES=75 > > DB_CONNECTIONS=20 > > REGRESS_DURATION=1 #HOURS > > REGRESS_DURATION_SEC=$((60*60*$REGRESS_DURATION)) > > > > DBT2 Command : > > ./dbt2-pgsql-create-db > > ./dbt2-pgsql-build-db -d $DBDATA -g -r -w $WAREHOUSES > > ./dbt2-run-workload -a pgsql -c $DB_CONNECTIONS -d > > $REGRESS_DURATION_SEC -w $WAREHOUSES -o $OUTPUT_DIR > > ./dbt2-pgsql-stop-db > > > > I am not able to understand the sar related graphs. Iostat,mpstat and vmstat > > results are similar but > > sar results are strange. I tried to explore the dbt2 source code to find > > out the how graphs are drawn and why sar results differ.DBT2.pm : 189 reads > > sar.out and parse it and consider 1 minute elapsed time between each record > > i.e. > > That is certainly a weakness in the logic of the perl modules in > plotting the charts accurately. I wouldn't be surprised if the other > stat tools suffer the same problem. > > Regards, > Mark _________________________________________________________________ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. https://signup.live.com/signup.aspx?id=60969 --_f4d244f6-548a-4dad-a3e6-508274af7dc1_ Content-Type: text/html; charset="windows-1256" Content-Transfer-Encoding: 8bit I am facing sar related issues on Redhat Enterprise Linux64 5.4 too (60G Ram, No Swap space, Xeon Processor).

sar -o /var/dbt2_data/PG/Output/driver/dbt2-sys1/sar_raw.out 60 204
    |___ sadc 60 205 -z /var/dbt2_data/PG/Output/driver/dbt2-sys1/sar_raw.out

It generates following sar data i.e.
….
….
03:52:43 AM      2.31
03:53:43 AM      2.31
03:54:43 AM      2.28
03:55:43 AM      2.31
03:56:43 AM      1.67
03:57:43 AM      0.29
03:58:43 AM      0.29
04:00:43 AM      0.30
04:04:00 AM      3.52
04:07:07 AM      0.30
04:09:36 AM      0.23
04:12:04 AM      0.36
04:14:25 AM      0.23
04:16:45 AM      0.26
04:19:10 AM      0.24
04:21:30 AM      0.38
04:23:55 AM      0.24
04:26:25 AM      0.35
04:28:48 AM      0.24
04:31:10 AM      0.27
04:33:40 AM      0.33
04:36:45 AM      0.41
04:39:12 AM      0.27
04:41:41 AM      0.26
04:44:11 AM      0.33
04:46:35 AM      0.25
04:49:06 AM      0.33
04:51:27 AM      0.27
04:53:56 AM      0.23
04:56:19 AM      0.36
04:58:43 AM      0.24
05:01:10 AM      0.35
05:03:43 AM    &nbs p; 0.33
05:06:53 AM      0.29
05:09:25 AM      0.23
….
….

To fix this issue I have modified the sysstat-9.1.2/sadc.c and replaced signal based pause (That is not real time) with "select" based pause. That fixed the issue. Thanks.

-------------------------------------------------------------------------------------------------------------------------------------------------------
sadc.c.patch
-------------------------------------------------------------------------------------------------------------------------------------------------------
--- sadc.c.org    2010-06-14 21:44:18.000000000 +0500
+++ sadc.c    2010-06-14 22:52:51.693211184 +0500
@@ -33,6 +33,10 @@
 #include <sys/stat.h>
 #include <sys/utsname.h>
 
+#include <sys/types.h>
+#include <sys/time.h>
+#include <time.h>
+
 #include "version.h"
 #include "sa.h"
 #include "rd_stats.h"
@@ -792,6 +796,15 @@
     }
 }
 
+void pause_new( void )
+{
+    struct timeval tvsel;
+    tvsel.tv_sec = interval;
+    tvsel.tv_usec = 0;
+
+    select( 0, NULL, NULL, NULL, &tvsel );
+}
+
 /*
  ************************************ ***************************************
  * Main loop: Read stats from the relevant sources and display them.
@@ -899,7 +912,7 @@
         }
 
         if (count) {
-            pause();
+            pause_new();
         }
 
         /* Rotate activity file if necessary */
-------------------------------------------------------------------------------------------------------------------------------------------------------

Best Regards,
Asif Naeem

> Date: Wed, 21 Apr 2010 18:10:35 -0700
> Subject: Re: [PERFORM] Dbt2 with postgres issues on CentOS-5.3þ
> From: markwkm@gmail.com
> To: anaeem.it@hotmail.com
> CC: pgsql-performance@postgresql.org
>
> 2010/4/20 MUHAMMAD ASIF <anaeem.it@hotmail.com>:
> > Hi,
> >
> > I am using dbt2 on Linux 64 (CentOS release 5.3 (Final)) . I have compiled
> > latest postgresql-8.4.3 code on the machine and run dbt2 against it. I am
> > little confused about the results. I ran dbt2 with the following
> > configuration i.e.
> >
> > DBT2 Options :
> >     WAREHOUSES=75
> >     DB_CONNECTIONS=20
> >     REGRESS_DURATION=1 #HOURS
> >     REGRESS_DURATION_SEC=$((60*60*$REGRESS_DURATION))
> >
> > DBT2 Command :
> >         ./dbt2-pgsql-create-db
> >         ./dbt2-pgsql-build-db -d $DBDATA -g -r -w $WAREHOUSES
> >         ./dbt2-run-workload -a pgsql -c $DB_CONNECTIONS -d
& gt; > $REGRESS_DURATION_SEC -w $WAREHOUSES -o $OUTPUT_DIR
> >         ./dbt2-pgsql-stop-db
> >
> > I am not able to understand the sar related graphs. Iostat,mpstat and vmstat
> > results are similar but
> > sar results are strange. I tried to explore the dbt2 source code to find
> > out the how graphs are drawn and why sar results differ.DBT2.pm : 189 reads
> > sar.out and parse it and consider 1 minute elapsed time between each record
> > i.e.
>
> That is certainly a weakness in the logic of the perl modules in
> plotting the charts accurately. I wouldn't be surprised if the other
> stat tools suffer the same problem.
>
> Regards,
> Mark

Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. Sign up now. --_f4d244f6-548a-4dad-a3e6-508274af7dc1_--