T610 C-states and turbo mode under Centos-5.3

[Hansjörg Maurer]

Hi

we are evaluation a Poweredge T610 with two  Intel X5570 and 6*4 GByte
1333Mhz RAM .

We are planing to build a small (M1000e with  M610 modules) based HPC
blade cluster and test with this evaluation system,
how our application scales with this new intel CPU's.

Each task uses 1.4GByte of memory and with the old FSB based Intel
architecture,
the limited memory bandwith makes quad core CPU's scale badly with our
application.

The T610  comes with turbo mode and cstates activated in bios
Our plan was, to run 1,2,4,8,16 (with HT enabled) on this system, and
test how it scales.

But even with one task, we got no reproducible results
If we start one task, the runtime varies between 201s and 370s (80% ) on
the system, which was complitly idle apart from this single task
We the tried to bind the task manualy to differnt CPU's, which leads to
the following runtimes

CPU Runtime
CPU0 227s
CPU1 201s
CPU2 364s
CPU3 369s
CPU4 .... (all  365 s)
The results now were reproduceble. and the runtime depends on the CPU,
the task runs

The numbering of the CPU's under linux is like:


NUM = CPU number reported by linux (cat /proc/cpuinfo)
PID: Processor-ID
CID: Core-ID

NUM PID CID
0 1 0
1 0 0
2 1 1
3 0 1
4 1 2
5 0 2
6 1 3
7 0 3
The fastest runtime could be achieved under processor 0 and core 0,
followed by  processor 1 and core 0.

We then checked the memory configuration, which seems to be ok (one
Module in each bank).
Then we went through the bios and deactivated some defaults.

When we deactivated c-states (CPU-Settings) (turbo-mode still activated)
, the runtimes are equal on all CPU's
at about 225s.
This is not as fast as the fastest runtime we achieved on processor 0
core 0.


The system runs under Centos-5.3 (x86_64)
I am unable to load the  acpi-cpufreq kernel module
Linux shows the same CPU MHz even if the system is completly idle

cat /proc/cpuinfo |grep MHz
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153
cpu MHz : 2926.153


Therefore I have some questions
- what could be the reason, that the runtime varies at about 80% if
c-states are enabled in bios?
- are turbo mode and c-states supported under RHEL-5.3 or centos-5.3 on
this new plattform?
We want the blade system to reduce the CPU cycles if it is idle due to
cost for power consumptions and we want
to it to use turbo-mode, if only 1 job is running on a cpu
- how can I monitor the c-states an turbo-mode state under Linux on this
plattform?


redards

Hansjörg








-- 
_________________________________________________________________

Deutsches Zentrum fuer Luft- und Raumfahrt e.V.
in der Helmholtz-Gemeinschaft

Institut fuer Robotik und Mechatronik

Dr. Hansjörg Maurer

LAN- und Systemmanager

Münchner Strasse 20
82234 Wessling
Germany 

Telefon: 08153/28-2431 
Telefax: 08153/28-1134

E-Mail: Hansjoerg.Maurer at dlr.de
Internet: http://www.robotic.dlr.de/

__________________________________________________________________


There are 10 types of people in this world, 
those who understand binary and those who don't.