OpenStack Folsom Quantum Devstack Installation Tutorial

OpenStack Folsom Quantum Devstack Installation Tutorial and Screencast

This is a quick guide that includes a diagram of a working reference architecture for installation of the OpenStack Folsom release using which includes the Quantum networking component using the DevStack installation bash script.

With the OpenStack Essex release, I was writing installers with Python for Linux bridging and the Quantum plugin. So far the OpenStack Folsom install is more complex due to the added options mainly from the Quantum networking plugin. A big feature of Quantum out of the box using this build with OpenvSwitch is it adds features such as layer2 multi-tenancy for path isolation with GRE tunnels and/or Vlan IDs.

This tutorial will install the packages as listed under Folsom below. Trying to keep the Python installer up to date is too much maintenance for me so switching to DevStack for people who get paid to do it. While installing by hand is good for initial builds, this release mainly due to the complexity of Quantum is trickier than Essex in my opinion. I think that’s some ironic commentary on SDN for those who track developments and try and separate reality, fear and myth on that front.

Figure 1. OpenStack Releases (Note Quantum is still in development and not stable until ‘Grizzly’ Q2Y2013.

[/crayon]
Figure 2. Vendor contributions and how they have differed from Essex to Folsom.

OpenStack Folsom Installation Pre-Requisites

I am using Ubuntu 12.04. Never hurts to patch and update but not required. Then we will download Devstack project from GitHub. Since the hooks for OpenvSwitch are coupled with KVM you will need physical hardware that supports hardware virtualization. Nested support is road-mapped and may already be integrated with HEAT in OpenStack for hypervisor hooks into the public cloud.

Check for hardware virtualization on Intel (Intel-VT) with:

Check for hardware virtualizationon AMD (AMD-V) with:

You should see VMX or SVM highlighted in red from the commands above if HW virtualization is supported.

Download and install git. Then clone the devstack repo on github.

Here is what my network NIC configurations look like in /etc/network/interfaces. We want the port to be up in promiscuous mode. Think of Snort or OpenVPN interface configuration 🙂

Above is the /etc/network/interfaces file that determines the “ifconfig” output. Other handy binaries ethtool “apt-get install ethtool” to determine if you have link to a NIC, e.g. “ethtool eth0” would return a Boolean on link detected: yes|no and tcpdump is installed by default, “tcpdump host 172.16.1.100 -i eth0” would promiscuously listen for the host specified on interface eth0.

DevStack Localrc File

Add the following in the devstack directory and name it “localrc”. It is the same directory that contains the “stack.sh” shell script. The localrc file contains all of your build parameters to pass to the script. Finding the right combination took way too many hours but here is what is used in the screencast at the bottom of the post.

Be sure to read the script and understand what other parameters are available. This will create unique Vlan IDs between the host and Hypervisor. If for example you wanted to use GRE tunnels instead you would add the line “ENABLE_TENANT_TUNNELS=True” to the configuration. This is the key to the install. If you have two NICs on a host this should work for you.

If you downloaded devstack git source a a few days or even hours before you execute it you may consider updating the installer. Assuming you allowed devstack to install into the default /opt/stack/ directory, running this will update the latest code merge.

Run ./stack.sh to Install OpenStack Folsom

With your localrc file in the same directory run ./stack.sh. Once the installation is done run the following to pull-down and import the Amazon EC2 Ubuntu Cloud Image. I had a terrible time with what gets pulled down with the script and you will notice at the end I actually get errors on tokens from Glance but everything still works fine EXCEPT for the Cirros image. You can always test if your image is working by looking at it in the Dashboard through VNC. Nova will report a host up even if it is sitting their with a broken image trying to pxi boot.

Your shell environmentals need to have similar values to the following. This is service authentication. Merely pasting it into your bash shell or adding it to localrc for persistency will give you CLI permissions.

[/crayon]
Figure 3. Check the VM if you are troubleshooting. Nova still reports it up even if it is in this state.

The Glance error looks something like this. It’s like the SERVICE_TOKEN variable isnt getting passed even though it is defined in keystone somewhere. Did not spend any time on it since I want Ubuntu Precise Cloud Image. You can build windows images with this tutorial.

Before you boot a host you need to do one thing that is not automated. You have to add your fixed_range or back-end NIC to the br-int OpenvSwitch bridge. You are taking that physical interface eth1 and binding it to br-int. Think of it as a locally significant Vlan with no tags being inserted (Probably worst explanation ever). That physical host still does not have any addresses on it.

Folsom Quantum L3-agent

The L3-agent will take care of arp-proxying the hosts. The eth1 host is up promiscuously. That is how if two hosts in the same tenant/bridge/subnet would always show the same arp entry for neighboring VMs. Example, VM Host1 at 192.168.1.10/24 and VM Host2 at 192.168.1.11/24 and a gateway of 192.168.1.1. Pinging 192.168.1.1 from Host1 would give you the same mac address in the arp entry of the gateway as you would see if your pinged Host2. This is because all traffic is funneled through the L3-agent. The datapaths (Layer2 path) is kept isolated by two mechanisms, one being Vlan tagging from the VM to the Physical host and the other option being GRE tunnels from a VM to Physical. This was not possible prior to the OpenStack Folsom release OpenStack Essex. Essex used default Linux bridging.

Folsom leverages advanced vSwitch features in the case OpenvSwitch in conjunction with IPTables filtering and DNSMASQ to deliver self provisioning L2/L3/L4 header for policy application from flow based forwarding. Elegant solution and more on networking in the next section with some drawings.

Note that when you spawn VM from the DevStack install as shown in the video. I use the “demo” group and in the networking tab select “private”.

Video 1. Quick walk through of post script operations.

OpenStack Quantum Networking

[/crayon]
Figure 4. Data Path (Layer 2) isolation is achieved between hypervisor and VM host with Vlans a GRE tunnels. Those encapsulations break out at the hypervisor or could be picked up by emerging SDN/Data Center orchestration networking.

The concepts are pretty cool. Our future is apparently going to be reliant on DNSMASQ, IPTables or plug-ins like NVP to build state and forwarding policy, see Cisco VSG and Nexus 1k, its all the same concepts, forwarding instantiation. Remember that Quantum won’t be released as stable until Grizzle in spring of 2013. There are quite a few pieces that are not baked into Horizon (Dashboard) since Horizon development froze for Folsom release. Take a look at the blueprint if interested in seeing upcoming features. Floating addresses is not supported via Horizon so you need to add those via the API or CLI.

[/crayon]
Figure 5. Here are where some of the DevStack parameters fit into the topology and how OpenStack networking from Hypervisor down looks.

Figure 7. Quantum Namespaces are the path isolation components. Getting familiar with ‘ip netns exec’ commands will be part of a deep dive.

Uninstall or Reinstall the Devstack OpenStack Environment

Depending on if I ran into problems or not I will often delete installed packages and directories to get as vanilla an install as I can get. Since this will be physical hosts the snapshot image isn’t available. Something along the following will delete some main packages and delete some directories that will get reinstalled as soon as you run the ./stack.sh script again. Remember to just unload the Devstack script and processes like quantum, nova, glance etc run ./unstack.sh. That will not stop dependencies like your hypervisor or vSwtich which in this case is KVM/libvirt-bin and OpenvSwitch.

Potential DevStack Folsom Errors to Troubleshoot

Horizon Error: Template error Dashboard with slug “nova” is not registered. Just delete /opt/stack/horizon/ and pull it done again and the error will clean up. I didn’t troubleshoot Django to try and see why since just rebuilding the directory fixes it up.