poor network performance with KVM virtualisation (CentOS with virtio)
emsearcy at gmail.com
Fri Jul 15 16:32:01 CDT 2011
On Jul 10, 2011, at 1:50 AM, Mihai Cristea wrote:
> we noticed that our Dell R815 is pretty slow in handling multiple VMs when using Linux/KVM. In our setup we used CentOS 5.6 in host, and CentOS in guests, too.
> The slowness we noticed especially on the network traffic between VMs (iperf reports 20Mbps udp traffic!!! between 2 VMs). Again, we instantiated only 2 VMs on this 48cores machine, and that's the performance we get with UDP; TCP gets better, something around 1.2Gbps - but still bad for this powerful & expensive machine.
> We tried on latest CentOS 5.6 kernels and also we did update to the latest linux kernels (2.6.38), using latest virtIO drivers, etc. We tried most of tips we've found on Internet/KVM-forums regarding this topic.
> What should I do to improve the NIC performance? What do we do wrong?
> I'd like to know if we need specific drivers for linux kernel for this machine (especially on the motherboard chipset that handles I/Os, or on the Broadcom NICs)?
Your Broadcom NICs shouldn't be relevant, VM-to-VM traffic would be going over the virtual bridge (br0 or similar), with the NIC being configured like a switch port on the bridge for bridge edge egress/ingress traffic.
Do you get better results with identical configuration on other hardware, so it's not a CentOS 5.6 or KVM/qemu/libvirt configuration issue? There's plenty of other things on the non-hardware side, like are you using virtio instead NIC emulation for your guest? I don't do too much KVM (still using Xen) but there's a fair amount of KVM knowledge on the centos-virt mailing list, so you might want to ask there unless you're pretty sure it's an R815 issue.
More information about the Linux-PowerEdge