[Crowbar] learning crowbar within kvm: nova and network barclamp ?

Pierre Amadio pierre.amadio at canonical.com
Tue Jul 17 04:37:06 CDT 2012


Hi there !

I am trying to learn crowbar in a kvm based environment. This means i
have a large machine (a Dell C6100, model XS23-TY35 with 70G of ram)
where i can run nearly as many vms i need.

I was able to deploy essex on a couple of vms hosted on this box using
the default networking barclamp:

In the kvm hypervisor, the following network were defined:

crowbar124  192.168.124.0/24
isolated    192.168.123.0/24

<network>
  <name>crowbar124</name>
  <uuid>c46a86d4-34f0-ddfa-1118-922b78d375ff</uuid>
  <forward mode='nat'/>
  <bridge name='virbr1' stp='on' delay='0' />
  <mac address='52:54:00:26:F2:BB'/>
  <ip address='192.168.124.1' netmask='255.255.255.0'>
  </ip>
</network>
and
<network>
  <name>isolated</name>
  <uuid>a90c07ed-8c1f-6b5f-f774-1505cbb4c5d2</uuid>
  <bridge name='virbr2' stp='on' delay='0' />
  <mac address='52:54:00:08:37:BA'/>
  <ip address='192.168.123.0' netmask='255.255.255.0'>
  </ip>
</network>

the crowbar admin node was having 192.168.124.10 set on eth0.
4 nodes plugged to the crowbar124 network on eth0 and the isolated one
on eth1 were running mysql/keystone/glance/nova-dashboard/nova (mutliple
controller/multiple compute and multiple volume).

Then, I wanted to try to use different network settings, but fail to
have something working for each test i performed.

Ideally, i would like to reach the following:

###############################
admin          192.168.120.0/24
public         192.168.121.0/24
nova_floating: 192.168.121.128/25 (sub range of public)
nova_fixed:    192.168.123.0/24
storage            192.168.124.0/24
################################

I dont use vlans at all.
Openstack network mode will be flatdhcp.

My first problem is about deciding how many nic to use on each nodes.
I do not plan to use nic bonding, and i do not understand exactly what
can be done, and what can not.

According to
https://github.com/dellcloudedge/crowbar/wiki/Network--barclamp

there are 3 existing mode for the network barclamp:

"""
single, dual, or team.

The mode specifies a base line goal of the system. Single means to use
the first interface (usually eth0) as the only interface for all
networks. VLANs and bridges will be added on top of this single
interface. Dual means that the first interface should be used as the
admin interface and the second interface should be used for the other
interfaces. Team means that the first two interfaces should be bonded
together and all VLANs run over it.
"""

I first try to create 2 kvm network, one for the admin network
(192.168.120.0), one for all the others, plugged the admin node eth0 to
the admin network, and eth1 to the other kvm network; changed the
/etc/network/interfaces of the admin node so eth0 is 192.168.120.2.

I also edited
/opt/dell/barclamps/network/chef/data_bags/crowbar/bc-template-network.json
so that the network mode was dual and the other network were changed.

I never manage to get a usable crowbar this way: running "./install
crowbar125.mydomain.com" took between 24 and 48 hours, and i ended up
with my crowbar admin node not being listed in the existing nodes.

There was no pxe boot server replying to request on eth0 (nor on eth1
for that matter).

Has any one of you successfully played with crowbar in a kvm environment
? If so, i would be interested to know more about the kvm networks you
defined, the vms connectivity to those network, and the content of the
bc-template-network.json file that you used. (oh, btw, 'i'm melmoth on
irc :-) just ping me there if you do run crowbar in kvm).

So, i temporarily gave up the idea of using the dual mode and kept using
single mode. I change the bc-template-network.json (see attachement)
accordingly.

I ended up with a crowbar install that seems to be working.

I was able to see my admin node, and to deploy mysql/keystone and glance.

Things started to fail when i deployed nova.

I deployed it this way:

network:    	

    ha_enabled: true
    num_networks: 1
    dhcp_enabled: true
    network_size: 256
    allow_same_net_traffic: false
    tenant_vlans: false

nova-multi-compute node2 node3
nova-multi-controller node1
nova-multi-volumes node1

All nodes have eth0 plugged in the 192.168.120.0/24 kvm network, and
eth1 to eth4 plugged in another kvm network (where i was expecting to
run all the other networks).

node1 also have an additional disk so it can run as a nova-volume.

This barclamp installation fail, and all my nodes ends up in this
spinning yellow state

I cannot connect on node1,2 and 3 with the network anymore, but i have
access to their console. It turns out its network settings have been
changed.

Before applying the nova barclamp, i must have had something like:

eth0  -> 192.168.120.something
no bridges, no addresses on the other nic.

(because i was able to ssh into the nodes without problem)

Now, the only interface being up is a bridge called breth0 with an
address in 192.168.123.0/24 (bridged into eth0)

I suppose my bc-template-network.json is set in such a way that this
problem occurs, but i have no idea how to:

- tell crowbar what i want as far as network are concerned.
- how many kvm network i should create for this, and into wich nic to
plug them.

Any help appreciated. Thanks !

-------------- next part --------------
A non-text attachment was scrubbed...
Name: bc-template-network.json
Type: application/json
Size: 6764 bytes
Desc: not available
Url : http://lists.us.dell.com/pipermail/crowbar/attachments/20120717/d561ece5/attachment.bin 


More information about the Crowbar mailing list