pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Dual Xeon + HW RAID question
16+ messages / 9 participants
[nested] [flat]

* Dual Xeon + HW RAID question
@ 2003-07-10 17:43 alexandre arruda paes :: aldeia digital <alexandre@aldeiadigital.com.br>
  2003-07-11 10:55 ` Re: Dual Xeon + HW RAID question Rod Taylor <rbt@rbt.ca>
  0 siblings, 1 reply; 16+ messages in thread

From: alexandre arruda paes :: aldeia digital @ 2003-07-10 17:43 UTC (permalink / raw)
  To: pgsql-performance

Hi,

I have this machine with a 10 million records:
* Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI , Adaptec 2110S,
RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0 - 2.4.18

The database is mirrored with contrib/dbmirror in a P4 1 Gb Ram + IDE

If a disk failure occurs, I can use the server in the mirror.

I will format the main server in this weekend and I have seen in the list
some people that recomends a Software RAID instead HW.

I think too remove the RAID 5 and turn a RAID 1 for data in 2 HDs.
SO, WAL and swap in the thrid HD.

My questions:

1) I will see best disk performance changing the disk layout like above
2) HyperThreading really improve a procces basead program, like postgres

Thank´s for all

Alexandre





^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-10 17:43 Dual Xeon + HW RAID question alexandre arruda paes :: aldeia digital <alexandre@aldeiadigital.com.br>
@ 2003-07-11 10:55 ` Rod Taylor <rbt@rbt.ca>
  0 siblings, 0 replies; 16+ messages in thread

From: Rod Taylor @ 2003-07-11 10:55 UTC (permalink / raw)
  To: alexandre arruda paes :: aldeia digital <alexandre@aldeiadigital.com.br>; +Cc: pgsql-performance

> 2) HyperThreading really improve a procces basead program, like postgres

I've not seen the results of this type of measurement posted, so really
couldn't say.

Attachments:

  signature.asc (application/pgp-signature, 186B) download

^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
@ 2003-07-12 18:25 Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-14 11:19 ` Re: Dual Xeon + HW RAID question Andrew Sullivan <andrew@libertyrms.info>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  0 siblings, 2 replies; 16+ messages in thread

From: Nikolaus Dilger @ 2003-07-12 18:25 UTC (permalink / raw)
  To: alexandre@aldeiadigital.com.br; +Cc: pgsql-performance

Alexandre,

Since you want the fastest speed I would do the 2 data
disks in RAID 0 (striping) not RAID 1 (mirroring).

If you would care about not loosing any transactions
you would keep all 3 disks in RAID 5.

Don't know the answer to the Hyperthreading question. 
Why don't you run a test to find out?

Regards,
Nikolaus

On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), "alexandre
arruda paes :: aldeia digital" wrote:

> 
> Hi,
> 
> I have this machine with a 10 million records:
> * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI
,
> Adaptec 2110S,
> RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0
-
> 2.4.18
> 
> The database is mirrored with contrib/dbmirror in a P4
> 1 Gb Ram + IDE
> 
> If a disk failure occurs, I can use the server in the
> mirror.
> 
> I will format the main server in this weekend and I
> have seen in the list
> some people that recomends a Software RAID instead HW.
> 
> I think too remove the RAID 5 and turn a RAID 1 for
> data in 2 HDs.
> SO, WAL and swap in the thrid HD.
> 
> My questions:
> 
> 1) I will see best disk performance changing the disk
> layout like above
> 2) HyperThreading really improve a procces basead
> program, like postgres
> 
> Thank´s for all
> 
> Alexandre
> 
> 
> 
> ---------------------------(end of
> broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send
> an appropriate
>       subscribe-nomail command to
> majordomo@postgresql.org so that your
>       message can get through to the mailing list
> cleanly



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
@ 2003-07-14 11:19 ` Andrew Sullivan <andrew@libertyrms.info>
  1 sibling, 0 replies; 16+ messages in thread

From: Andrew Sullivan @ 2003-07-14 11:19 UTC (permalink / raw)
  To: pgsql-performance

On Sat, Jul 12, 2003 at 11:25:14AM -0700, Nikolaus Dilger wrote:
> Alexandre,
> 
> Since you want the fastest speed I would do the 2 data
> disks in RAID 0 (striping) not RAID 1 (mirroring).

Note that RAID 0 buys you nothing at all in redundancy.  So if the
point is to be able to recover from a disk failure, you need 1 (or
some combination of 0 and 1, or 5).

A


-- 
----
Andrew Sullivan                         204-4141 Yonge Street
Liberty RMS                           Toronto, Ontario Canada
<andrew@libertyrms.info>                              M2P 2A8
                                         +1 416 646 3304 x110




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
@ 2003-07-21 05:09 ` SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 18:07   ` Re: Dual Xeon + HW RAID question alexandre paes :: aldeia digital <alexandre@aldeiadigital.com.br>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  1 sibling, 2 replies; 16+ messages in thread

From: SZUCS Gábor @ 2003-07-21 05:09 UTC (permalink / raw)
  To: pgsql-performance

Alexandre,

I missed your orig. post, but AFAIK multiprocessing kernels will handle HT
CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4
CPUs.

This way, I don't think HT would improve any single query (afaik no postgres
process uses more than one cpu), but overall multi-query performance has to
improve.

----- Original Message ----- 
From: "Nikolaus Dilger" <nikolaus@dilger.cc>
Sent: Saturday, July 12, 2003 8:25 PM


Alexandre,

Since you want the fastest speed I would do the 2 data
disks in RAID 0 (striping) not RAID 1 (mirroring).

If you would care about not loosing any transactions
you would keep all 3 disks in RAID 5.

Don't know the answer to the Hyperthreading question.
Why don't you run a test to find out?

Regards,
Nikolaus

On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), "alexandre
arruda paes :: aldeia digital" wrote:

>
> Hi,
>
> I have this machine with a 10 million records:
> * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI
,
> Adaptec 2110S,
> RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0
-
> 2.4.18
>
> The database is mirrored with contrib/dbmirror in a P4
> 1 Gb Ram + IDE
>
> If a disk failure occurs, I can use the server in the
> mirror.
>
> I will format the main server in this weekend and I
> have seen in the list
> some people that recomends a Software RAID instead HW.
>
> I think too remove the RAID 5 and turn a RAID 1 for
> data in 2 HDs.
> SO, WAL and swap in the thrid HD.
>
> My questions:
>
> 1) I will see best disk performance changing the disk
> layout like above
> 2) HyperThreading really improve a procces basead
> program, like postgres
>
> Thank´s for all
>
> Alexandre




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
@ 2003-07-21 18:07   ` alexandre paes :: aldeia digital <alexandre@aldeiadigital.com.br>
  1 sibling, 0 replies; 16+ messages in thread

From: alexandre paes :: aldeia digital @ 2003-07-21 18:07 UTC (permalink / raw)
  To: pgsql-performance

SZUCS,

In my tests, I don´t a great performance enhacement with HT.

I suspect that my problem resides on I/O performance. I will
wait for a best moment to resinstall the system with other
disk configurations and then I will report here.


Thanks for all replys!

Alexandre

> Alexandre,
>
> I missed your orig. post, but AFAIK multiprocessing kernels will handle HT
> CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4
> CPUs.
>
> This way, I don't think HT would improve any single query (afaik no
> postgres
> process uses more than one cpu), but overall multi-query performance has
> to
> improve.
>
> ----- Original Message -----
> From: "Nikolaus Dilger" <nikolaus@dilger.cc>
> Sent: Saturday, July 12, 2003 8:25 PM
>
>
> Alexandre,
>
> Since you want the fastest speed I would do the 2 data
> disks in RAID 0 (striping) not RAID 1 (mirroring).
>
> If you would care about not loosing any transactions
> you would keep all 3 disks in RAID 5.
>
> Don't know the answer to the Hyperthreading question.
> Why don't you run a test to find out?
>
> Regards,
> Nikolaus
>
> On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), "alexandre
> arruda paes :: aldeia digital" wrote:
>
>>
>> Hi,
>>
>> I have this machine with a 10 million records:
>> * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI
> ,
>> Adaptec 2110S,
>> RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0
> -
>> 2.4.18
>>
>> The database is mirrored with contrib/dbmirror in a P4
>> 1 Gb Ram + IDE
>>
>> If a disk failure occurs, I can use the server in the
>> mirror.
>>
>> I will format the main server in this weekend and I
>> have seen in the list
>> some people that recomends a Software RAID instead HW.
>>
>> I think too remove the RAID 5 and turn a RAID 1 for
>> data in 2 HDs.
>> SO, WAL and swap in the thrid HD.
>>
>> My questions:
>>
>> 1) I will see best disk performance changing the disk
>> layout like above
>> 2) HyperThreading really improve a procces basead
>> program, like postgres
>>
>> Thank´s for all
>>
>> Alexandre
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org
>




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
@ 2003-07-21 19:07   ` Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  1 sibling, 1 reply; 16+ messages in thread

From: Bruce Momjian @ 2003-07-21 19:07 UTC (permalink / raw)
  To: SZUCS Gábor <surrano@mailbox.hu>; +Cc: pgsql-performance

SZUCS Gábor wrote:
> Alexandre,
> 
> I missed your orig. post, but AFAIK multiprocessing kernels will handle HT
> CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4
> CPUs.
> 
> This way, I don't think HT would improve any single query (afaik no postgres
> process uses more than one cpu), but overall multi-query performance has to
> improve.

When you use hyperthreading, each virtual cpu runs at 70% of a full CPU,
so hyperthreading could be slower than non-hyperthreading.  On a fully
loaded dual cpu system, you are looking at 2.8 cpu's (0.70 * 4), while
if it isn't loaded, you are looking at slowing down if you are only
using 1 or 2 cpu's.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
@ 2003-07-22 07:23     ` Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  0 siblings, 1 reply; 16+ messages in thread

From: Mindaugas Riauba @ 2003-07-22 07:23 UTC (permalink / raw)
  To: pgsql-performance


> > I missed your orig. post, but AFAIK multiprocessing kernels will handle
HT
> > CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4
> > CPUs.
> >
> > This way, I don't think HT would improve any single query (afaik no
postgres
> > process uses more than one cpu), but overall multi-query performance has
to
> > improve.
>
> When you use hyperthreading, each virtual cpu runs at 70% of a full CPU,
> so hyperthreading could be slower than non-hyperthreading.  On a fully
> loaded dual cpu system, you are looking at 2.8 cpu's (0.70 * 4), while
> if it isn't loaded, you are looking at slowing down if you are only
> using 1 or 2 cpu's.

  Virtual cpus are not running at 70% of real cpus :). Slowdown will happen
if
scheduler will run 2 processes on the same real cpu. And I read that there
are
patches for Linux kernel to fix that. Sooner rather than later they will
appear
in Linus kernel.

  Mindaugas




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
@ 2003-07-22 16:26       ` Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  0 siblings, 1 reply; 16+ messages in thread

From: Bruce Momjian @ 2003-07-22 16:26 UTC (permalink / raw)
  To: Mindaugas Riauba <mind@bi.lt>; +Cc: pgsql-performance

Mindaugas Riauba wrote:
> 
> > > I missed your orig. post, but AFAIK multiprocessing kernels will handle
> HT
> > > CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4
> > > CPUs.
> > >
> > > This way, I don't think HT would improve any single query (afaik no
> postgres
> > > process uses more than one cpu), but overall multi-query performance has
> to
> > > improve.
> >
> > When you use hyperthreading, each virtual cpu runs at 70% of a full CPU,
> > so hyperthreading could be slower than non-hyperthreading.  On a fully
> > loaded dual cpu system, you are looking at 2.8 cpu's (0.70 * 4), while
> > if it isn't loaded, you are looking at slowing down if you are only
> > using 1 or 2 cpu's.
> 
>   Virtual cpus are not running at 70% of real cpus :). Slowdown will happen
> if
> scheduler will run 2 processes on the same real cpu. And I read that there
> are
> patches for Linux kernel to fix that. Sooner rather than later they will
> appear
> in Linus kernel.

Right, I simplified it.  The big deal is whether the OS favors the
second real CPU over one of the virtual CPU's on the same die --- by
default, it doesn't.  Ever if it did work perfectly, you are talking
about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
@ 2003-07-22 17:10         ` SZUCS Gábor <surrano@mailbox.hu>
  2003-07-22 17:35           ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  2003-07-22 17:38           ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  0 siblings, 2 replies; 16+ messages in thread

From: SZUCS Gábor @ 2003-07-22 17:10 UTC (permalink / raw)
  To: pgsql-performance

"by default" -- do you mean there is a way to tell Linux to favor the second
real cpu over the HT one? how?

G.
------------------------------- cut here -------------------------------
----- Original Message ----- 
From: "Bruce Momjian" <pgman@candle.pha.pa.us>
Sent: Tuesday, July 22, 2003 6:26 PM
Subject: Re: [PERFORM] Dual Xeon + HW RAID question


> Right, I simplified it.  The big deal is whether the OS favors the
> second real CPU over one of the virtual CPU's on the same die --- by
> default, it doesn't.  Ever if it did work perfectly, you are talking
> about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much.




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
@ 2003-07-22 17:35           ` Jord Tanner <jord@indygecko.com>
  2003-07-22 17:39             ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  1 sibling, 1 reply; 16+ messages in thread

From: Jord Tanner @ 2003-07-22 17:35 UTC (permalink / raw)
  To: SZUCS Gábor <surrano@mailbox.hu>; +Cc: pgsql-performance

The Linux 2.6 kernel will have the ability to set CPU affinity for
specific processes. There is a patch for the 2.4 kernel at
http://www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity

RedHat 9 already has support for CPU affinity build in.

The July 2003 issue of Linux Journal includes a little C program (on
page 20) that gives you a shell level interface to the CPU affinity
system calls, so you can dynamically assign processes to specific CPUs.
I haven't tried it, but it looks very cool (my only SMP machine is in
production, and I don't want to mess with it). If you try it out, please
share your experiences with the list.


Jord Tanner
Independent Gecko Consultants

On Tue, 2003-07-22 at 10:10, SZUCS Gábor wrote:
> "by default" -- do you mean there is a way to tell Linux to favor the second
> real cpu over the HT one? how?
> 
> G.
> ------------------------------- cut here -------------------------------
> ----- Original Message ----- 
> From: "Bruce Momjian" <pgman@candle.pha.pa.us>
> Sent: Tuesday, July 22, 2003 6:26 PM
> Subject: Re: [PERFORM] Dual Xeon + HW RAID question
> 
> 
> > Right, I simplified it.  The big deal is whether the OS favors the
> > second real CPU over one of the virtual CPU's on the same die --- by
> > default, it doesn't.  Ever if it did work perfectly, you are talking
> > about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much.
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
-- 
Jord Tanner <jord@indygecko.com>




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-22 17:35           ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
@ 2003-07-22 17:39             ` Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:54               ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  0 siblings, 1 reply; 16+ messages in thread

From: Bruce Momjian @ 2003-07-22 17:39 UTC (permalink / raw)
  To: Jord Tanner <jord@indygecko.com>; +Cc: SZUCS Gábor <surrano@mailbox.hu>; pgsql-performance


But CPU affinity isn't realated to hyperthreading, as far as I know. 
CPU affinity tries to keep processes on the same cpu in case there is
still valuable info in the cpu cache.

---------------------------------------------------------------------------

Jord Tanner wrote:
> The Linux 2.6 kernel will have the ability to set CPU affinity for
> specific processes. There is a patch for the 2.4 kernel at
> http://www.kernel.org/pub/linux/kernel/people/rml/cpu-affinity
> 
> RedHat 9 already has support for CPU affinity build in.
> 
> The July 2003 issue of Linux Journal includes a little C program (on
> page 20) that gives you a shell level interface to the CPU affinity
> system calls, so you can dynamically assign processes to specific CPUs.
> I haven't tried it, but it looks very cool (my only SMP machine is in
> production, and I don't want to mess with it). If you try it out, please
> share your experiences with the list.
> 
> 
> Jord Tanner
> Independent Gecko Consultants
> 
> On Tue, 2003-07-22 at 10:10, SZUCS G?bor wrote:
> > "by default" -- do you mean there is a way to tell Linux to favor the second
> > real cpu over the HT one? how?
> > 
> > G.
> > ------------------------------- cut here -------------------------------
> > ----- Original Message ----- 
> > From: "Bruce Momjian" <pgman@candle.pha.pa.us>
> > Sent: Tuesday, July 22, 2003 6:26 PM
> > Subject: Re: [PERFORM] Dual Xeon + HW RAID question
> > 
> > 
> > > Right, I simplified it.  The big deal is whether the OS favors the
> > > second real CPU over one of the virtual CPU's on the same die --- by
> > > default, it doesn't.  Ever if it did work perfectly, you are talking
> > > about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much.
> > 
> > 
> > ---------------------------(end of broadcast)---------------------------
> > TIP 8: explain analyze is your friend
> -- 
> Jord Tanner <jord@indygecko.com>
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
>       subscribe-nomail command to majordomo@postgresql.org so that your
>       message can get through to the mailing list cleanly
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-22 17:35           ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  2003-07-22 17:39             ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
@ 2003-07-22 17:54               ` Jord Tanner <jord@indygecko.com>
  2003-07-22 18:50                 ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  0 siblings, 1 reply; 16+ messages in thread

From: Jord Tanner @ 2003-07-22 17:54 UTC (permalink / raw)
  To: Bruce Momjian <pgman@candle.pha.pa.us>; +Cc: SZUCS Gábor <surrano@mailbox.hu>; pgsql-performance

On Tue, 2003-07-22 at 10:39, Bruce Momjian wrote:
> But CPU affinity isn't realated to hyperthreading, as far as I know. 
> CPU affinity tries to keep processes on the same cpu in case there is
> still valuable info in the cpu cache.
> 

It is true that CPU affinity is designed to prevent the dump of valuable
CPU cache. My thought is that if you are trying to prevent CPU
contention, you could use CPU affinity to prevent 2 postmaster processes
from running simultaneously on the same die. Am I out to lunch here?
I've not worked with CPU affinity before, so I'm not familiar with the
intimate details.
 

-- 
Jord Tanner <jord@indygecko.com>




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-22 17:35           ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  2003-07-22 17:39             ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:54               ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
@ 2003-07-22 18:50                 ` Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 19:18                   ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  0 siblings, 1 reply; 16+ messages in thread

From: Bruce Momjian @ 2003-07-22 18:50 UTC (permalink / raw)
  To: Jord Tanner <jord@indygecko.com>; +Cc: SZUCS Gábor <surrano@mailbox.hu>; pgsql-performance

Jord Tanner wrote:
> On Tue, 2003-07-22 at 10:39, Bruce Momjian wrote:
> > But CPU affinity isn't realated to hyperthreading, as far as I know. 
> > CPU affinity tries to keep processes on the same cpu in case there is
> > still valuable info in the cpu cache.
> > 
> 
> It is true that CPU affinity is designed to prevent the dump of valuable
> CPU cache. My thought is that if you are trying to prevent CPU
> contention, you could use CPU affinity to prevent 2 postmaster processes
> from running simultaneously on the same die. Am I out to lunch here?
> I've not worked with CPU affinity before, so I'm not familiar with the
> intimate details.

I guess you could but it is the backends that use the cpu.  I don't
think manually specifying affinity will work for most applications.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-22 17:35           ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  2003-07-22 17:39             ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:54               ` Re: Dual Xeon + HW RAID question Jord Tanner <jord@indygecko.com>
  2003-07-22 18:50                 ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
@ 2003-07-22 19:18                   ` Jord Tanner <jord@indygecko.com>
  0 siblings, 0 replies; 16+ messages in thread

From: Jord Tanner @ 2003-07-22 19:18 UTC (permalink / raw)
  To: Bruce Momjian <pgman@candle.pha.pa.us>; +Cc: SZUCS Gábor <surrano@mailbox.hu>; pgsql-performance

On Tue, 2003-07-22 at 11:50, Bruce Momjian wrote:
> Jord Tanner wrote:
> > On Tue, 2003-07-22 at 10:39, Bruce Momjian wrote:
> > > But CPU affinity isn't realated to hyperthreading, as far as I know. 
> > > CPU affinity tries to keep processes on the same cpu in case there is
> > > still valuable info in the cpu cache.
> > > 
> > 
> > It is true that CPU affinity is designed to prevent the dump of valuable
> > CPU cache. My thought is that if you are trying to prevent CPU
> > contention, you could use CPU affinity to prevent 2 postmaster processes
> > from running simultaneously on the same die. Am I out to lunch here?
> > I've not worked with CPU affinity before, so I'm not familiar with the
> > intimate details.
> 
> I guess you could but it is the backends that use the cpu.  I don't
> think manually specifying affinity will work for most applications.

This is beating a dead horse, but I'll take one more kick at it.

CPU affinity is defined by a bit mask, so multiple processors can be
selected. It is also inherited by child processes, so assigning CPU 0
and CPU 2 (which I assume would be on different dies in a dual processor
hyper-threading system) to the parent postmaster should prevent CPU
contention with respect to the postgres backend. 

I would be very interested to see if any advantage could be gained by a
combination of multiple HT processors and cpu affinity over multiple
non-HT processors. Yet Another Performance Testing To Do (YAPTTD)!

-- 
Jord Tanner <jord@indygecko.com>




^ permalink  raw  reply  [nested|flat] 16+ messages in thread

* Re: Dual Xeon + HW RAID question
  2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
  2003-07-21 05:09 ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
  2003-07-21 19:07   ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 07:23     ` Re: Dual Xeon + HW RAID question Mindaugas Riauba <mind@bi.lt>
  2003-07-22 16:26       ` Re: Dual Xeon + HW RAID question Bruce Momjian <pgman@candle.pha.pa.us>
  2003-07-22 17:10         ` Re: Dual Xeon + HW RAID question SZUCS Gábor <surrano@mailbox.hu>
@ 2003-07-22 17:38           ` Bruce Momjian <pgman@candle.pha.pa.us>
  1 sibling, 0 replies; 16+ messages in thread

From: Bruce Momjian @ 2003-07-22 17:38 UTC (permalink / raw)
  To: SZUCS Gábor <surrano@mailbox.hu>; +Cc: pgsql-performance

SZUCS Gábor wrote:
> "by default" -- do you mean there is a way to tell Linux to favor the second
> real cpu over the HT one? how?

Right now there is no way the kernel can tell which virtual cpu's are on
each physical cpu's, and that is the problem.  Once there is a way,
hyperthreading will be more useful, but even then, it doesn't double
your CPU throughput, just increases by 40%.


> > Right, I simplified it.  The big deal is whether the OS favors the
> > second real CPU over one of the virtual CPU's on the same die --- by
> > default, it doesn't.  Ever if it did work perfectly, you are talking
> > about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much.
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 8: explain analyze is your friend
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



^ permalink  raw  reply  [nested|flat] 16+ messages in thread


end of thread, other threads:[~2003-07-22 19:18 UTC | newest]

Thread overview: 16+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-07-10 17:43 Dual Xeon + HW RAID question alexandre arruda paes :: aldeia digital <alexandre@aldeiadigital.com.br>
2003-07-11 10:55 ` Rod Taylor <rbt@rbt.ca>
2003-07-12 18:25 Re: Dual Xeon + HW RAID question Nikolaus Dilger <nikolaus@dilger.cc>
2003-07-14 11:19 ` Andrew Sullivan <andrew@libertyrms.info>
2003-07-21 05:09 ` SZUCS Gábor <surrano@mailbox.hu>
2003-07-21 18:07   ` alexandre paes :: aldeia digital <alexandre@aldeiadigital.com.br>
2003-07-21 19:07   ` Bruce Momjian <pgman@candle.pha.pa.us>
2003-07-22 07:23     ` Mindaugas Riauba <mind@bi.lt>
2003-07-22 16:26       ` Bruce Momjian <pgman@candle.pha.pa.us>
2003-07-22 17:10         ` SZUCS Gábor <surrano@mailbox.hu>
2003-07-22 17:35           ` Jord Tanner <jord@indygecko.com>
2003-07-22 17:39             ` Bruce Momjian <pgman@candle.pha.pa.us>
2003-07-22 17:54               ` Jord Tanner <jord@indygecko.com>
2003-07-22 18:50                 ` Bruce Momjian <pgman@candle.pha.pa.us>
2003-07-22 19:18                   ` Jord Tanner <jord@indygecko.com>
2003-07-22 17:38           ` Bruce Momjian <pgman@candle.pha.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox