10 gigabit Ethernet - Add on question

[Kuba Ober]

On Tuesday 08 January 2008, Matthias Saou wrote:
> Kuba Ober wrote :
> > On Monday 31 December 2007, Robert D. Holtz - Lists wrote:
> > > How do you topple the memory bus/cup/peripheral bus restraints to even
> > > push past ~2gbps per host, in a practical host environment?
> >
> > 1gbps is AFAIK less than 100 megabytes/s if you factor in overheads.
> >
> > So 10 gbps is say 800 megabytes/s, practically, at a high link
> > utilization. It should be relatively easy to achieve that on modern
> > hardware. The memory bandwidths are a few times beyond that, same goes
> > for DMA bandwidths. With current PCI Express-supporting chipsets, if you
> > do memory-to-network transfers (just for benchmarking), the other bus
> > devices won't even notice, and all the CPU will see is less leftover
> > memory bandwidth.
> >
> > One DDR2 PC-3200 memory stick will happily transfer way more than 1GB/s
> > to 4 PCIe lanes, on relatively standard hardware. I tried.
>
> This is true for "raw" network traffic, because limits quickly depend
> on what and how you transfer. My testings on some brand new dual quad
> core PE1950 III servers (E5410 @ 2.33GHz w/ 6MB cache) show for
> instance that an rsync/ssh transfer from one to the other maxes out at
> 50MB/s because that corresponds to one ssh process taking 100% of one
> of the 8 cores. This is one of those cases where pure MHz power
> would be more useful than having so many cores :-)

That's obvious enough. You can parallelize ssh connections, all it takes is a 
short program to split the data onto 2 or more connections. If you want it, I 
can post one.

Cheers, Kuba