Open vSwitch GRE Tunnel Configuration

Open vSwitch GRE Tunnel Screencast

Watch video on YouTube

We continue to destroy our nice clean data centers to facilitate poor application architectures that do not scale in a layer 3 fashion. We continue to come up with new technologies using old architectures to turn our networks into flat data centers under the guise of “cloud mobility”. Ok sorry, I got that out of my system.

Open vSwitch and Data Center Overlays

Open vSwitch is a virtual switch or virtual Ethernet bridge (VEB) becoming quite popular and being backed by many vendors as the alternative to the virtual switches in some of the productized hypervisors like Hyper-V, VMware etc. Overlays are not new concepts, but are being used in new use cases. It is an early mechanism to uncouple the virtual from the physical. What remains to be seen, is whether this is a bridge until hardware becomes generalized or the new standard.

The OVS developers is still ironing out the control portion of OVS, with that will come the scale of L3 addressing and multipathing to each Tunnel End Point (TEP). The current code supports the GRE key lookup and forwarding while NVGRE and VXlan have at least taken a stab at the control plane, unfortunately through multicast for broadcast flooding L Next-gen ASICs will and Ethernet controllers will open the door to more plates of spaghetti.

Figure 1.  Generic look at where the headers are being modified.

I am still on the fence on where a TEP should reside, on a physical top of rack end of row network device or reside in the hypervisor. The networking guy in me would prefer that to be an S-tag in a pseudowire, but I can’t help but lean towards anything in software, moving forward to provide programmatic scale. We are on the path of the programmatic abstraction to being migrated into our network operating systems, but that is a big ship that will presumably take a while to steer.

Enough gibber on to an L2GRE configuration on an Open vSwitch. I did this because I was surprised to not see a GRE cookbook on the Open vSwitch cookbook page. http://openvswitch.org/support/config-cookbooks/

BTW why the kitchen cooking trend these days? What happened to the good old how-to? Not cloudy enough, that needed abstraction also. Do we change RTFM to RTFC? I blame the Opscode Chef Openstack installer.

Open vSwitch Installation Tutorials

  • First install Open vSwitch on a couple of nodes. The following links will help you get started:
gre openvswitch

Figure 2. Here is the topology of the configuration.

Open vSwitch 1 configuration

sudo ifconfig eth1 0
sudo ovs-vsctl add-br br1
sudo ovs-vsctl add-br br2
sudo ovs-vsctl add-port br1 eth0
sudo ifconfig br1 192.168.1.155 netmask 255.255.255.0
sudo ifconfig br2 10.1.1.1 netmask 255.255.255.0
sudo ovs-vsctl add-port br2 gre0 -- set interface gre0 type=gre options:remote_ip=192.168.1.152

Open vSwitch 2 configuration

Hypervisor 2 Configuration:

sudo ifconfig eth1 0
sudo ovs-vsctl add-br br1
sudo ovs-vsctl add-br br2
sudo ovs-vsctl add-port br1 eth0
sudo ifconfig br1 192.168.1.152 netmask 255.255.255.0
sudo ifconfig br2 10.1.1.2 netmask 255.255.255.0
sudo ovs-vsctl add-port br2 gre0 -- set interface gre0 type=gre options:remote_ip=192.168.1.155
  • Add KVM tap VIF script
/etc/ovs-ifup and /etc/ovs-ifdown

#Paste in the following with br1 = equalling the bridge you made in ovs-vsctl. Since we want the VM on the bridge that will get tunneled we are not using br0. Create this script for provisioning.

nano /etc/ovs-ifup
#+++++++++++++++++++++++++++++++
#!/bin/sh
switch='br2'
/sbin/ifconfig $1 0.0.0.0 up
ovs-vsctl add-port ${switch} $1
#+++++++++++++++++++++++++++++++
chmod +x /etc/ovs-ifup

Then one for ifdown to de-provision.

nano /etc/ovs-ifdown
#+++++++++++++++++++++++++++++++
#!/bin/sh
switch=’br2’
/sbin/ifconfig $1 0.0.0.0 down
ovs-vsctl del-port ${switch} $1
#+++++++++++++++++++++++++++++++
chmod +x /etc/ovs-ifdown
  • Now spin up your VM. I use the following to boot Ubuntu off CD. The VM instance will tap br1 (bridge1) on the 10.1.1.0/24 network.
kvm -m 512 -net nic,macaddr=12:34:52:CC:CC:10 \
 -net tap,script=/etc/ovs-ifup,downscript=/etc/ovs-ifdown \
 -cdrom /home/brent/images/ubuntu-11.10-desktop-i386.iso &

Attach OpenvSwitch to a Floodlight openFlow Controller

Hypervisor 1
#Install the Floodlight Openflow Controller
git clone git://github.com/floodlight/floodlight.git
cd floodlight
ant
java -jar target/floodlight.jar
#Add OVS bridges
sudo ovs-vsctl set-controller br1 tcp:192.168.1.155:6633
sudo ovs-vsctl set-controller br2 tcp:192.168.1.155:6633
Hypervisor 2
#Install the Floodlight Openflow Controller
git clone git://github.com/floodlight/floodlight.git
cd floodlight
ant
java -jar target/floodlight.jar
#Add OVS bridges
sudo ovs-vsctl set-controller br1 tcp:192.168.1.155:6633
sudo ovs-vsctl set-controller br2 tcp:192.168.1.155:6633
Some commands to view the configuration and ports:
ubuntu-12:brent# ovs-appctl fdb/show bra (generate traffic ie ping as table age out after 300 seconds)
 port  VLAN  MAC                Age
    9     0  12:74:52:cc:cc:10    0 (VM 1)
   12     0  12:34:52:cc:cc:10    0 (VM 2)
openvswitch vlan

Figure 2. Now we have flattened out these two endpoints whether in the same data center or across the city/state that can be used for a live workload migration (e.g. vMotion).

openvswitch gre tunnels

Figure 3. There is some scale to where this can go, for either provisioning elastic compute nodes or migrating, live migrations or just fixing broken applications.