Memory performance on PE R610 (Adv ECC vs Optimizer)

[JACOB_LIBERMAN at Dell.com]

Hi Stephen,

If its memory bandwidth you're after, populate 1 DIMM per channel per socket across both sockets.  (6 DIMMs total)

On an R610 with 6 1333 MHz UDIMMs you should expect stream bandwidth of ~36 GB/s. (BIOS 1.0.4 and 1.1.4, 8 threads)

Copy	36577
Scale	36212
Add	34232
Triad	35240

I have heaps of performance data if you need design recommendations for a particular application.

Thanks, Jacob

> -----Original Message-----
> From: linux-poweredge-bounces at lists.us.dell.com [mailto:linux-
> poweredge-bounces at lists.us.dell.com] On Behalf Of Stephen Dowdy
> Sent: Friday, May 29, 2009 12:19 PM
> To: linux-poweredge-Lists
> Subject: FYI: Memory performance on PE R610 (Adv ECC vs Optimizer)
> 
> I knew the Nehalem's Integrated Memory Controller (IMC) had some
> wildly variant performance characteristics based upon a large
> number of criteria, but i was pretty surprised to actually test the
> performance of an As-Shipped 8GB config on the R610 in Advanced ECC
> mode versus reconfiguring it to Optimizer mode.
> 
> A good reference on the Nehalem's memory configuration issues is at:
> http://www.delltechcenter.com/page/04-08-2009+-
> +Nehalem+and+Memory+Configurations
> 
> Since the Dell quote configuration web app only lets you select a
> small number of memory configurations from the possible universe of
> configurations, should you want an R610 with only 8GB, you have to
> select it in Advanced ECC mode. If you don't need that extra level
> of fault-correction, you will definitely want to reconfigure the
> memory to run in Optimizer mode.
> 
> You need to re-arrange DIMMs (check the R610 hardware owners manual)
> into an optimizer mode configuration *AND* change the BIOS setting.
> (it appears that the new 1.1.4 BIOS may auto-select the "correct"
> setting and disallow manual changes??)
> 
> Anyway, running a STREAM memory benchmark on an R610 with a single
> processor and 8GB in Advanced ECC vs Optimizer shows a nearly
> 50% improvement in memory bandwidth:  (two tests each)
> (i haven't gotten around to doing a tri-channel optimizer config
> run)
> 
> (Advanced ECC mode) [ 4x 2GB DIMMs ]
> Function      Rate (MB/s)   Avg time     Min time     Max time
> Copy:        5026.9518       0.1022       0.1019       0.1032
> Scale:       5103.7723       0.1005       0.1003       0.1010
> Add:         5546.7697       0.1388       0.1385       0.1392
> Triad:       5497.6567       0.1399       0.1397       0.1404
> 
> Function      Rate (MB/s)   Avg time     Min time     Max time
> Copy:        5073.8188       0.1010       0.1009       0.1010
> Scale:       5100.7174       0.1005       0.1004       0.1006
> Add:         5633.3843       0.1365       0.1363       0.1368
> Triad:       5613.1721       0.1369       0.1368       0.1371
> 
> 
> (Optimizer mode)  [ 4x 2GB DIMMs (dual-chan)]
> Function      Rate (MB/s)   Avg time     Min time     Max time
> Copy:        7387.3101       0.0693       0.0693       0.0694
> Scale:       7345.7673       0.0698       0.0697       0.0698
> Add:         7560.9054       0.1016       0.1016       0.1016
> Triad:       7570.2341       0.1015       0.1014       0.1018
> 
> Function      Rate (MB/s)   Avg time     Min time     Max time
> Copy:        7386.8019       0.0694       0.0693       0.0694
> Scale:       7347.5518       0.0698       0.0697       0.0699
> Add:         7551.1040       0.1019       0.1017       0.1028
> Triad:       7575.9137       0.1015       0.1014       0.1016
> 
> I *presume* these test results would apply to the R710 as well as a
> dual-cpu configuration.
> 
> --stephen
> --
> Stephen Dowdy  -  Systems Administrator  -  NCAR/RAL
> 303.497.2869   -  sdowdy at ucar.edu        -
> http://www.ral.ucar.edu/~sdowdy/
> 
> _______________________________________________
> Linux-PowerEdge mailing list
> Linux-PowerEdge at lists.us.dell.com
> https://lists.us.dell.com/mailman/listinfo/linux-poweredge
> Please read the FAQ at http://lists.us.dell.com/faq