FYI: Memory performance on PE R610 (Adv ECC vs Optimizer)

[Stephen Dowdy]

I knew the Nehalem's Integrated Memory Controller (IMC) had some
wildly variant performance characteristics based upon a large
number of criteria, but i was pretty surprised to actually test the
performance of an As-Shipped 8GB config on the R610 in Advanced ECC
mode versus reconfiguring it to Optimizer mode.

A good reference on the Nehalem's memory configuration issues is at:
http://www.delltechcenter.com/page/04-08-2009+-+Nehalem+and+Memory+Configurations

Since the Dell quote configuration web app only lets you select a
small number of memory configurations from the possible universe of
configurations, should you want an R610 with only 8GB, you have to
select it in Advanced ECC mode. If you don't need that extra level
of fault-correction, you will definitely want to reconfigure the
memory to run in Optimizer mode.

You need to re-arrange DIMMs (check the R610 hardware owners manual)
into an optimizer mode configuration *AND* change the BIOS setting.
(it appears that the new 1.1.4 BIOS may auto-select the "correct"
setting and disallow manual changes??)

Anyway, running a STREAM memory benchmark on an R610 with a single
processor and 8GB in Advanced ECC vs Optimizer shows a nearly
50% improvement in memory bandwidth:  (two tests each)
(i haven't gotten around to doing a tri-channel optimizer config
run)

(Advanced ECC mode) [ 4x 2GB DIMMs ]
Function      Rate (MB/s)   Avg time     Min time     Max time
Copy:        5026.9518       0.1022       0.1019       0.1032
Scale:       5103.7723       0.1005       0.1003       0.1010
Add:         5546.7697       0.1388       0.1385       0.1392
Triad:       5497.6567       0.1399       0.1397       0.1404

Function      Rate (MB/s)   Avg time     Min time     Max time
Copy:        5073.8188       0.1010       0.1009       0.1010
Scale:       5100.7174       0.1005       0.1004       0.1006
Add:         5633.3843       0.1365       0.1363       0.1368
Triad:       5613.1721       0.1369       0.1368       0.1371


(Optimizer mode)  [ 4x 2GB DIMMs (dual-chan)]
Function      Rate (MB/s)   Avg time     Min time     Max time
Copy:        7387.3101       0.0693       0.0693       0.0694
Scale:       7345.7673       0.0698       0.0697       0.0698
Add:         7560.9054       0.1016       0.1016       0.1016
Triad:       7570.2341       0.1015       0.1014       0.1018

Function      Rate (MB/s)   Avg time     Min time     Max time
Copy:        7386.8019       0.0694       0.0693       0.0694
Scale:       7347.5518       0.0698       0.0697       0.0699
Add:         7551.1040       0.1019       0.1017       0.1028
Triad:       7575.9137       0.1015       0.1014       0.1016

I *presume* these test results would apply to the R710 as well as a
dual-cpu configuration.

--stephen
-- 
Stephen Dowdy  -  Systems Administrator  -  NCAR/RAL
303.497.2869   -  sdowdy at ucar.edu        -  http://www.ral.ucar.edu/~sdowdy/