Configuring VXLan and GRE tunnels on OpenvSwitch
Update: VXLan Encapsulation
I have done a couple of GRE tunnel how-tos using OpenvSwitch (OVS). I had been itching to give VXLan a spin in OVS so why not ferret out someones tree on GitHub. I believe VXLan is still scheduled to officially release soon in OpenvSwitch. So here are the steps for installing, configuring tunnels on OpenvSwitch with VXLan and GRE encapsulations. At the end we will compare some of the protocols with difference MTU sizes. The results were interesting I think (for a nerd). We will be installing and then configure both GRE and VXLan encapsulated tunnels using Open vSwitch.
I like seeing some collaboration of really smart people from different companies as displayed in the GPL below.
* Copyright (c) 2012 Cisco Systems Inc.
* Distributed under the terms of the GNU GPL version 2.
* * Significant portions of this file may be copied from parts of the Linux
* kernel, by Linus Torvalds and others. */
Figure 1. Example of how tunnels can be leveraged as overlays.
By the way I should probably disclaim now that huge Layer2 networks do not scale and huge Layer3 networks do. Host count in a broadcast domain/Vlan/network should be kept to a reasonable 3 digit number. Cisco is quick to point out that OTV is the solution over WAN’s for extending Layer 2 networks with OTV as the solution to extend Layer 2 Vlans. That said Overlays are flat out required to overcome Vlan number limitations and have lots of potential with programmatic orchestration.
Figure 2. OVS punts the first packet to user land for the forwarding decision and passes the data path back to the data path in the kernel for subsequent packets in the flow. slow path for the first packet then fast path for the rest.
I dd some simple Iperf tests at the end of the post using different MTU values of 1500 and 9000 byte MTUs. The numbers are kind of fun to look at from the result. For a really nice analysis take a look at Martin’s post at Network Heresy on comparing STT, Linux Bridging and GRE. I am going to spin up some VMs on the VXLan tunnel later this week and measure the speeds a little closer and see how GRE and VXLan stack up to one another from hosts using overlays. I just ran some Iperfs from the hypervisor itself rather than VMs. OVS also supports Capwap encapsulation that performs mac in GRE which is rather slick. Wonder why we do not hear much about that. I am going to dig in when I get back to home from the road later next week.
Figure 3. Lab Setup basically has a fake interface up with Br1. Real world Br1 would have VMs tapped on it. The video uses br1 and br2 but it got to be confusing for people so I changed it to br0 and br1 to match most peoples eth0 = br0 NIC naming in ifconfig.
Quick Video of the fast install below.
If you would prefer to install using packages check out this post. It will install an older version that may or may not support VXLAN framing depending on the timeline. It will support GRE.
OpenvSwitch Configure from Packages and Attaching to a Floodlight OpenFlow Controller →
For those familiar with the build you can just paste the following in your bash shell as root. To walk through the install skip the following snippet.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
apt-get update apt-get install -y git python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config linux-headers-`uname -r` git clone git://openvswitch.org/openvswitch cd openvswitch ./boot.sh ./configure --with-linux=/lib/modules/`uname -r`/build make && make install insmod datapath/linux/openvswitch.ko touch /usr/local/etc/ovs-vswitchd.conf mkdir -p /usr/local/etc/openvswitch ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema ovsdb-server /usr/local/etc/openvswitch/conf.db \ --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --private-key=db:SSL,private_key \ --certificate=db:SSL,certificate \ --bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file ovs-vsctl --no-wait init ovs-vswitchd --pidfile --detach ovs-vsctl show |
Open vSwitch System Preperation
This is on two boxes with 64 cores and Broadcom Corporation NetXtreme II BCM57810 10G Nics to a random 10G TOR switch.
Install dependencies
|
1 2 |
apt-get update apt-get install python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config |
Download the Open vSwitch latest build
|
1 2 |
git clone git://openvswitch.org/openvswitch cd openvswitch |
Compile OpenvSwitch From Source
|
1 2 3 4 5 6 |
./boot.sh ./configure --with-linux=/lib/modules/`uname -r`/build make make install #Load the OVS Kernel Module insmod datapath/linux/openvswitch.ko |
|
1 2 3 4 |
#If you are running a stripped version of Nix like an EC2 cloud image and get an error along the lines of this: configure: error: source dir /lib/modules/3.2.0-23-virtual/build doesn't exist #Pull down the headers for your kernel. sudo apt-get install linux-headers-`uname -r` |
Initial Open vSwitch Configuration
|
1 2 3 |
touch /usr/local/etc/ovs-vswitchd.conf mkdir -p /usr/local/etc/openvswitch ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema |
Run the following commands, note some dashes are two ‘- -’ e.g. – -remote=db & – -private-key
Start ovsdb-server, this stores the config into a file that is persistent even after restarts.
|
1 2 3 4 5 6 |
ovsdb-server /usr/local/etc/openvswitch/conf.db \ --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --private-key=db:SSL,private_key \ --certificate=db:SSL,certificate \ --bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file |
|
1 2 3 4 5 6 7 8 |
#Only need to run this the first time. ovs-vsctl --no-wait init #Start vswitch ovs-vswitchd --pidfile --detach #Verify the kernel module(s) in case you didn't earlier and get errors. #lsmod | grep br #brcompat 13512 0 #openvswitch 98196 1 brcompat |
*note* “brcompat” is depreciated since the OVS upstream. Output should just be “openvswitch” as a loaded kernel module. If they are not there try loading again and check your path to the kernel module.
|
1 |
insmod datapath/linux/openvswitch.ko |
At this point you have a fucntioning vanilla OVS install. Output should look something like this.
|
1 2 3 4 5 |
ovs-vsctl show b6d574d7-5582-4cc0-93e5-a90cf0eb0a38 root@demo-139:/home/ubuntu/ovs-vxlan# ovs-vsctl --version ovs-vsctl (Open vSwitch) 1.8.90 Compiled Aug 19 2012 06:23:36 |
Configure Linux Networking
I have one NIC (eth0) on the same LAN segment/network/vlan.
We are attaching eth0 to br1 and applying an IP to the bridge interface.
We are attaching an IP to br1. br1 is the island that we are building a tunnel for hosts to connect on. Without the VXLAN tunnel, the two br1 interfaces should not be able to ping one another. Note: This is being setup on the same subnet, it is important to keep in mind that the VXLAN framing will allow for the tunnel to be established over disparate networks. E.g. can be done over the Internet etc.
Configuration for Host 1
|
1 2 3 4 5 6 7 8 |
#Host 1 Configuration ovs-vsctl add-br br0 ovs-vsctl add-br br1 ovs-vsctl add-port br0 eth0 ifconfig eth0 0 && ifconfig br0 192.168.1.10 netmask 255.255.255.0 route add default gw 192.168.1.1 br0 ifconfig br1 10.1.2.10 netmask 255.255.255.0 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11 |
Configuration for Host 2
|
1 2 3 4 5 6 7 8 |
#Host 1 Configuration ovs-vsctl add-br br0 ovs-vsctl add-br br1 ovs-vsctl add-port br0 eth0 ifconfig eth0 0 && ifconfig br0 192.168.1.11 netmask 255.255.255.0 route add default gw 192.168.1.1 br0 ifconfig br1 10.1.2.11 netmask 255.255.255.0 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.10 |
If you have issues getting the bridge built you may need to kill the OVS processes and restart them depending on your step order.
Your Linux routing table and ifconfig should now look something like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
root@ub64:/home/brent/openvswitch# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 br0 10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br1 192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 root@ub64:/home/brent/openvswitch# ifconfig br0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::285f:3ff:fea2:dafc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:622 errors:0 dropped:0 overruns:0 frame:0 TX packets:54 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:171756 (171.7 KB) TX bytes:3832 (3.8 KB) br1 Link encap:Ethernet HWaddr ea:34:7d:49:e7:49 inet addr:10.1.2.10 Bcast:10.1.2.255 Mask:255.255.255.0 inet6 addr: fe80::c0a7:aaff:fe90:29dc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:12 errors:0 dropped:0 overruns:0 frame:0 TX packets:13 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:896 (896.0 B) TX bytes:986 (986.0 B) eth0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet6 addr: fe80::20c:29ff:fe65:fd82/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1222493 errors:0 dropped:0 overruns:0 frame:0 TX packets:53824 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:653399936 (653.3 MB) TX bytes:4089412 (4.0 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:7088 errors:0 dropped:0 overruns:0 frame:0 TX packets:7088 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:537609 (537.6 KB) TX bytes:537609 (537.6 KB) |
Troubleshooting Open vSwitch Installation
An error was sent to me by someone that looked like this below, restarting the OVS procs will clear it up:
|
1 2 3 4 5 6 7 8 9 10 |
Aug 19 06:34:04 demo-139 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl del-br br1 Aug 19 06:34:04 demo-139 ovs-vswitchd: 00026|dpif|WARN|failed to enumerate system datapaths: No such file or directory Aug 19 06:34:04 demo-139 ovs-vswitchd: 00027|bridge|INFO|ovs-vswitchd (Open vSwitch) 1.8.90 Aug 19 06:34:20 demo-139 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl add-br br1 Aug 19 06:34:20 demo-139 ovs-vswitchd: 00028|dpif|WARN|failed to enumerate system datapaths: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00029|dpif|WARN|failed to create datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00030|ofproto_dpif|ERR|failed to open datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00031|ofproto|ERR|failed to open datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00032|bridge|ERR|failed to create bridge br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00033|bridge|INFO|ovs-vswitchd (Open vSwitch) 1.8.90 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
#Find the OVS PIDs from PS ps -ea | grep ovs 16369 ? 00:00:00 ovsdb-server 16372 ? 00:00:00 ovs-vswitchd 16373 ? 00:00:00 ovs-vswitchd 16375 ? 00:00:00 ovs_workq #Terminate them kill 16369 kill 16372 kill 16375 #Then restart the OVS server ovsdb-server /usr/local/etc/openvswitch/conf.db \ --remote=punix:/usr/local/var/run/openvswitch/db.sock \ --remote=db:Open_vSwitch,manager_options \ --private-key=db:SSL,private_key \ --certificate=db:SSL,certificate \ --bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file #Start vswitchd (JSON DB) ovs-vswitchd --pidfile --detach |
Build a GRE Tunnel in Open vSwitch
|
1 2 |
ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11 |
Adjust MTUs with % ifconfig <interface e.g. br0 eth0 etc> mtu 9000
#(Remember your switch needs to have >= your MTU.
Figure 4. Output of GRE Tunnels running with 1500 byte MTUs.
Figure 5. Output of GRE Tunnels running with 9000 byte MTUs.
Now delete the GRE ports
|
1 |
ovs-vsctl del-port gre1 |
Configure the VXLan Tunnel
The difference here is the “type” specifying the tunnel encapsulation.
|
1 2 |
ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.11 |
Figure 6 Output of VXLan Tunnels running with 1500 byte MTUs.
Figure 7 Output of VXLan Tunnels running with 9000 byte MTUs.
Thanks for stopping by.
Great job! You are so dedicated and your passion for what you are doing is inspiring!
Hi Brent,
Looks like the config commands under the “Add the VXLan tunnel” section are incorrect (still refer to GRE, when they should be doing vxlan).
Thanks Dmitri !!! I double checked it this time lol.
I think you do mistake in “Configure Linux Network” part. Must be br2 and eth2 above “#ifconfig eth2 0″ line.
Thanks Nerijus, Not sure why I had eth2 on there at all since the lab only uses one physical interface per host other than the bridges. Br1 carries the traffic to the two TEPs and Br2 are virtual islands where the VMs would reside that are totally Hypervisor software networking. Thanks for the help and catching that.
Cheers!
Thank you so much for these examples. It makes picking up new technologies so much easier!
My pleasure Evan! Thanks for the feedback, never sure if it makes any sense or not when I stare at it for a few hours putting it together lol. Let me know if you have any issues.
Cheers!
I have tried testing VXLAN on OVS with my own implementation of VXLAN.
I could receive the ping request from OVS and could send back the proper ping reply from my side with expected port of 4341.
But OVS is sending ICMP destination port unreachable message. Is it required to configure anything on OVS side for this.?
Please help.
Hi NS. Paste in ‘ovs-vsctl show’ ‘ifconfig’ and ‘route -n’ and we can take a look.
I will try and get a chance to verify the current snapshot on Mestery’s GitHub for you and do a screen cap of it.
please find the snapshots for “ovs-vsctl show”,”ifconfig”,”route -n”
As i said earlier,”OVS is sending ICMP destination port unreachable message” in response to my ping response. The OVS is searching a hash table and is not able to find the port it received in VXLAN message. It seems to me that some configuration is missing for the vxlan tunnel regarding the destination port.
root@user-desktop:~# ovs-vsctl show
c354223a-a435-403c-b33b-e045ce0f9c58
Bridge “br1″
Port “eth2″
Interface “eth2″
Port “br1″
Interface “br1″
type: internal
Bridge “br2″
Port “vx1″
Interface “vx1″
type: vxlan
options: {out_key=”310″, remote_ip=”192.168.4.1″}
Port “br2″
Interface “br2″
type: internal
———————————————-
root@user-desktop:~# ifconfig
br1 Link encap:Ethernet HWaddr 00:11:95:fc:8d:62
inet addr:192.168.4.2 Bcast:192.168.4.255 Mask:255.255.255.0
inet6 addr: fe80::211:95ff:fefc:8d62/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6 errors:0 dropped:0 overruns:0 frame:0
TX packets:115 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:448 (448.0 B) TX bytes:22940 (22.9 KB)
br2 Link encap:Ethernet HWaddr 22:28:bc:b6:64:4a
inet addr:192.168.6.22 Bcast:192.168.6.255 Mask:255.255.255.0
inet6 addr: fe80::2028:bcff:feb6:644a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:55 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:9962 (9.9 KB)
eth1 Link encap:Ethernet HWaddr 00:16:76:4b:91:dc
inet addr:192.168.90.64 Bcast:192.168.91.255 Mask:255.255.254.0
inet6 addr: fe80::216:76ff:fe4b:91dc/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:346769 errors:0 dropped:0 overruns:0 frame:0
TX packets:28058 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:78130470 (78.1 MB) TX bytes:4896867 (4.8 MB)
eth2 Link encap:Ethernet HWaddr 00:11:95:fc:8d:62
inet6 addr: fe80::211:95ff:fefc:8d62/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1144 errors:0 dropped:0 overruns:0 frame:0
TX packets:2959 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:96602 (96.6 KB) TX bytes:3573286 (3.5 MB)
Interrupt:22 Base address:0xc000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:18 errors:0 dropped:0 overruns:0 frame:0
TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1352 (1.3 KB) TX bytes:1352 (1.3 KB)
———————————————-
root@user-desktop:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.91.254 0.0.0.0 UG 100 0 0 eth1
192.168.90.0 0.0.0.0 255.255.254.0 U 0 0 0 eth1
192.168.4.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.6.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
I am actually having the same problem as you. I hooked wireshark up to my interfaces to check it and it was quite strange that vx1 was receiving and replying to the ARP packet but on the terminal it kept on showing the same message.
Would love to know if you’ve found some solution to this
Hi Brent
have you tried programming the VXlAN pipeline in the OVS using floodlight or any other Openflow 1.0 controller?
Can you shed some light on how these 1.0 rules might look like assuming that the ARPs are statically provisioned in each VM (learning not needed)for sake of brevity
you could consider the same example wherein you have shown the usage of VXLAN using static OVS commands
cheers
cole
So glad that I stumbled upon your blog. It is an excellent effort Brent. I am a newbie to SDN & tunneling. Have been studying for a few weeks only and I must say, you’re blog is very very helpful
I wanted to ask you if you knew what the difference between VXLAN patch released by Ben in Oct. 2011 and one you took from GIT is? I have been trying to implement that patch by Ben for some time and facing quite a lot of difficulties
Hi Farrukh, I am glad that helped, thats great. Gosh I dont have a clue what the difference between than and now is other than this effort was headed up by @mestery from Cisco in conjunction with Nicira and I am guessing Ben. The listserv might be your best bet.
Cheers,
-Brent
Thank you for that very quick reply. Much appreciated.
Mestery has cleaned up the code a little bit, made a few new header files. I am trying to implement his patch now. Thanks for the help. =)
Regards.
You rock Brent.
have configured gre tunnel using the steps above, but not seeing any traffic on iperf. can u tell me
Hmm is it pinging for you? Thanks.
Thanks. Very helpful.
What is the current state of “STT code to play with? “
Was just thinking that my self. Last time I pestered they said they would know in 4-6 weeks. Its been 6
Hi Brent,
Good post! Thanks!
Do you know if it is possible to run OVS in one of the VM’s without using nested VM architecture?
Hi Anna, you sure can. Matter of fact thats how I would build a tunnel today from a Public Cloud IaaS provider if I needed to tunnel back from there for whatever reason. Check out the OVS install instructions in this post.
http://networkstatic.net/public-cloud-network-as-a-service/
Its such a good question and point you make I should articulate it more. just add your IP to the br-int interface you make and that will use OVS as it’s address. From their you can add another bridge that acts as a dumb interface or really an object you can assign an IP to and point your default route for another NIC, VIF Loopback etc to. Let me know if you have problems before I get a post about it up later this week and I can elaborate better.
Thanks for stopping by and the great question!
Hello Brent, thanks very much for your informative post. I am facing a few issues when setting up the GRE tunnel between two VMs running openvswitch 1.7.1 (not the git branch you are referring to here). When I ping the br2 10.1.2.x IP address of the other VM, I see that the ARP messages encapsulated in GRE are reaching the other VM on the br1 interface, but I don’t see any responses from the other node with the MAC address of the local br2 interface. Can you suggest what to try next? thanks IP
Hi Brent,
Thanks for a great post. I was able to get VXLAN tunnels between VMs running. The throughput I am seeing however is at 6.1 Gbps with 9000 MTU on the ovsbridge (i should be seeing closer to 10Gbos). The openvswitch version I am using is the same, which makes me wonder if this has something to do with the kernel that I am running. Could you please give me some more details about your environment.
Yes. I have the same question. I am using 2 Dell T5500 Desktop. 8-core Intel Xeon 2.27GHz CPU. With Ubuntu 10.04, kernel 2.6.38.3. I connect 2 physical servers with a cross wire directly.
I use ovs-vxlan in https://github.com/mestery/ovs-vxlan
The throughput is only a bit more than half of the result in all cases in this blog.
What may be the other things that influence the throughput?
TCP variant?
Hi guys, Radhika, apologies for missing your question. Here is some info. If needed I can compile from the trunk and rerun the benchmarks for you guys. 2.6 is prior to OVS getting upstreamed in v3.3 so that could certainly have something to do with it.
Thanks,
-Brent
root@openstack3:~# uname -a
Linux openstack3 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
root@openstack3:~# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 45
Stepping: 7
CPU MHz: 2593.658
BogoMIPS: 5184.28
Virtualization: VT-x
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 20480K
NUMA node0 CPU(s): 0-7,16-23
NUMA node1 CPU(s): 8-15,24-31
root@openstack3:~# lspci | grep “Gig”
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
03:00.1 Ethernet controller: Broadcom Corporation NetXtreme II BCM57810 10 Gigabit Ethernet (rev 10)
root@openstack3:~# apt-get install netperf
root@openstack3:~# netperf -t TCP_STREAM -H 127.0.0.1 -l 10
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 (127.0.0.1) port 0 AF_INET : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
87380 16384 16384 10.00 21057.26
Hi Brent,
Thanks much for getting back on this! Yes, if you could share with us the same benchmarks against the latest ovs-vxlan branch, I’d be very grateful.
Here are my set up details:
uname -r
3.5.0-17-generic
modinfo openvswitch
filename: /lib/modules/3.5.0-17-generic/kernel/net/openvswitch/openvswitch.ko
version: 1.9.90
license: GPL
description: Open vSwitch switching datapath
srcversion: B833F5F3A240867927A7A45
depends:
vermagic: 3.5.0-17-generic SMP mod_unload modversions
Open vSwitch bridge info (MTU 9000):
ifconfig ovsbr
ovsbr Link encap:Ethernet HWaddr ae:56:8b:51:d6:4a
inet addr:10.0.113.3 Bcast:10.0.255.255 Mask:255.255.0.0
inet6 addr: fe80::ac56:8bff:fe51:d64a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:5199359 errors:0 dropped:0 overruns:0 frame:0
TX packets:2264022 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:37631493830 (37.6 GB) TX bytes:17375858808 (17.3 GB)
(other side is similar)
Physical link info (MTU 9000):
ifconfig p1p1
p1p1 Link encap:Ethernet HWaddr 90:e2:ba:26:88:c4
inet addr:15.0.113.3 Bcast:15.0.255.255 Mask:255.255.0.0
inet6 addr: fe80::92e2:baff:fe26:88c4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:4485908 errors:0 dropped:0 overruns:0 frame:0
TX packets:1993574 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:19248771222 (19.2 GB) TX bytes:5307066196 (5.3 GB)
(other side is similar)
Rx throughput (5.07 Gbps):
netperf -H 10.0.113.3 -f g
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.113.3 () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^9bits/sec
87380 65536 65536 10.00 5.07
Tx throughput (4.11 Gbps):
netperf -H 10.0.101.3 -f g
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.101.3 () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^9bits/sec
87380 65536 65536 10.00 4.11
Physical link throughput:
Rx (9.9Gbps):
netperf -H 15.0.113.3 -f g
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 15.0.113.3 () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^9bits/sec
87380 65536 65536 10.01 9.90
Tx(8.28 Gbps)
netperf -H 15.0.101.3 -f g
MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 15.0.101.3 () port 0 AF_INET
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^9bits/sec
87380 65536 65536 10.00 8.28
Since we really cannot play around much with offload settings of Open vSwitch, I do not know where the get back the missing throughput from. Your insights would be very helpful. I’ve also logged an issue at https://github.com/mestery/ovs-vxlan/issues/4 .
Also, I am running Ubuntu 12.10 on my server. Thanks much again!
Hi Guys,
I want to implement GTP tunneling on OVS. I have my C++ implementation of GTP but dont know how to integrate it into OVS and then test.
You may want to start by looking at tunnel.h first. That would help you define a new protocol in OVS. I do not know much details about GTP i.e. is it based on TCP, UDP etc? Or uses some other underlying protocol. So I don’t know where to point you next. Correct me if I am wrong here, I do not think I have seen any cpp files in OVS. It is based on C/Python.
Radhika and Wenfei, I finally got a chance to lab this up tonight and I am hitting a snag somewhere. Hopefully in the next day or two I will get a chance to troubleshoot it. Its 1am EST and I have meetings in the morning so I need to hit the sack. I will get the perf tests posted this week tho.
Farrukh thanks for helping Ghalib. Ghalib, have you taken a look at the Github? I was going the the code a couple weeks ago, but do not remember off the top of my head where the protocol was integrated. I will look this week.
Thanks guys,
-Brent
All numbers are w/ 9000byte MTU.
No tunnel:
[ 5] local 172.31.246.6 port 5001 connected with 172.31.246.7 port 55769
[ 5] 0.0-10.0 sec 11.5 GBytes 9.90 Gbits/sec
[ 4] local 172.31.246.6 port 5001 connected with 172.31.246.7 port 55770
[ 4] 0.0-10.0 sec 11.5 GBytes 9.90 Gbits/sec
GRE
[ 5] local 10.1.2.11 port 5001 connected with 10.1.2.12 port 35150
[ 5] 0.0-10.0 sec 1.83 GBytes 1.57 Gbits/sec
[ 4] local 10.1.2.11 port 5001 connected with 10.1.2.12 port 35151
[ 4] 0.0-10.0 sec 1.94 GBytes 1.66 Gbits/sec
VXLan
[ 4] local 10.1.2.11 port 5001 connected with 10.1.2.12 port 35161
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.1 sec 1.27 GBytes 1.08 Gbits/sec
[ 5] local 10.1.2.11 port 5001 connected with 10.1.2.12 port 35163
[ 5] 0.0-10.0 sec 1.13 GBytes 968 Mbits/sec
[ 4] local 10.1.2.11 port 5001 connected with 10.1.2.12 port 35164
[ 4] 0.0-10.1 sec 1.08 GBytes 919 Mbits/sec
sudo ifconfig eth0 mtu 9000
sudo ifconfig br2 mtu 9000
Noticed even though all of my ports are set with an MTU of 9k cannot get a frame >1500bytes thru.
root@openstack3:~/ovs-vxlan# ifconfig
br1 Link encap:Ethernet HWaddr e8:39:35:c4:92:20
inet addr:172.31.246.7 Bcast:172.31.246.255 Mask:255.255.255.0
inet6 addr: fe80::ea39:35ff:fec4:9220/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:14994643 errors:0 dropped:72 overruns:0 frame:0
TX packets:30261402 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2088400199 (2.0 GB) TX bytes:135203437061 (135.2 GB)
br2 Link encap:Ethernet HWaddr 3e:6c:21:50:78:4d
inet addr:10.1.2.12 Bcast:10.1.2.255 Mask:255.255.255.0
inet6 addr: fe80::3c6c:21ff:fe50:784d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:11451582 errors:0 dropped:0 overruns:0 frame:0
TX packets:859880 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:756727184 (756.7 MB) TX bytes:32818348274 (32.8 GB)
eth0 Link encap:Ethernet HWaddr e8:39:35:c4:92:20
inet6 addr: fe80::ea39:35ff:fec4:9220/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:12165982 errors:0 dropped:0 overruns:0 frame:0
TX packets:28392717 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1808756680 (1.8 GB) TX bytes:126984648851 (126.9 GB)
Interrupt:32 Memory:f6000000-f67fffff
eth1 Link encap:Ethernet HWaddr e8:39:35:c4:92:24
inet6 addr: fe80::ea39:35ff:fec4:9224/64 Scope:Link
UP BROADCAST RUNNING PROMISC MULTICAST MTU:9000 Metric:1
RX packets:49 errors:0 dropped:160 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6222 (6.2 KB) TX bytes:0 (0.0 B)
Interrupt:36 Memory:f4800000-f4ffffff
lo Link encap:Local Loopback
root@openstack3:~/ovs-vxlan# ping -s 8192 -M do 10.1.2.11
PING 10.1.2.11 (10.1.2.11) 8192(8220) bytes of data.
From 10.1.2.12 icmp_seq=1 Frag needed and DF set (mtu = 1500)
From 10.1.2.12 icmp_seq=1 Frag needed and DF set (mtu = 1500)
I will try and find some time to tshoot this weekend. My week is shot and I need to rack out tonight. Weird perf is this bad. Did see mestery tweeted the VXlan patch is submitted for upstream so may wait until then and check perf.
Thanks,
-Brent
Thanks for confirming Brent! The perf numbers do look pretty bad. Thanks for the headsup on the tweet. Looking forward to check it out when it gets in.
Do let us know if you find some settings that change the perf numbers whenever you get a chance.
I think I know what the performance problem may be. The patches I submitted upstream do not suffer performance issues compared to GRE. I’ll update my github with the same soon, once you see that commit, can you pull them and try them out?
Thanks,
Kyle
Hello Guys,
When I am Using GRE Tunnel in OVS, How I can get those messages in Wireshark, I am Running GRE and ping between two PC’s but Nothing of GRE is Shown in the WireShark.
Just ping Request and Response Shown there.
Hi Brent,
Thanks for all these great posts, they’ve been very usuful to setup some testing environments.
I tried to setup a scenario with vxlan connecting 3 different hypervisors. As I haven’t found any example of how to configure, I’ve tried the following implementation which is not working as expected, do you have an example configuration that you can share?
This is the one I’ve been trying, on Host 1:
Bridge br-int
–Port br-int
—-Interface br-int
——type:internal
–Port “eth0″
—-Interface “eth0″
Bridge “br2″
–Port “br2″
—-Interface “br2″
——type:interal
–Port vxlan
—-Interface vxlan
——type:vxlan
——options:{remote_ip=”192.168.1.1″}
–Port vxlan2
—-Interface vxlan2
——type:vxlan
——options:{remote_ip=”192.168.1.2.”}
I’ve read that vxlan on OVS in not implementing multicast, how does the protocol manage it? sending multi-unicast? or do we have to insert manually flow entries on each OVS?
Thanks!!
Hi Steve:
Yes, currently there is no multicast support in the VXLAN patches. The plan is that once the patch I’m working on goes upstream we can implement support for multicast learning in userspace.
Hope that helps!
Thanks,
Kyle
Hi Kyle,
Thanks for your answer, looking forward to test your patch once available! on how much time do you reckon it will be released?
Regarding my second question, do you know how to implement a multi tunnel configuration between different sites?
Thanks,
Steve
HI Brent,
How to setup STT in OVS. I need some document which can talk about STT setup in OVS.
Thanks,
Raju
Anyone having performance problems, please try the latest version of my patches from my github on the vxlan branch and let me know what you see. All perf issues should be fixed I believe.
Thanks,
Kyle
Hi Kyle,
Thanks for the update. I tried the latest branch, but I don’t see an improvement. Perhaps there is some config parameter that I am missing? I’d love to know insights on what are the knobs I could use to change the perf that I am seeing.
Here are my testbed details: physical links (both ends have 9000 MTU): 15.0.113.3 and 15.0.101.3
openvswitch with vxlan(both ends have 9000 MTU): 11.0.113.3 and 11.0.101.3
RX, Baseline (on physical link): 9.9Gbps
RX, VXLAN(openvswitch): 5.42 Gbps
TX, Baseline (on physical link): 8.32Gbps
TX, VXLAN(openvswitch): 4.57 Gbps
Output of ovs-ofctl show, ovs-dpctl show, ovs-vsctl show:
Server 1) ovs-ofctl show ovsbr
OFPT_FEATURES_REPLY (xid=0×1): dpid:00009e713aab6145
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(vx1): addr:5a:30:46:d7:73:b1
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
LOCAL(ovsbr): addr:9e:71:3a:ab:61:45
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0×3): frags=normal miss_send_len=0
ovs-dpctl show
system@ovs-system:
lookups: hit:3044610 missed:30 lost:0
flows: 0
port 0: ovs-system (internal)
port 1: ovsbr (internal)
port 2: vx1 (vxlan: remote_ip=15.0.101.3)
ovs-vsctl show
b7081fed-9be3-4243-893e-94b4b50211d8
Bridge ovsbr
Port ovsbr
Interface ovsbr
type: internal
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”15.0.101.3″}
Server 2) ovs-ofctl show ovsbr
OFPT_FEATURES_REPLY (xid=0×1): dpid:00001eb955408244
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(vx1): addr:5a:b1:02:dd:b7:15
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
LOCAL(ovsbr): addr:1e:b9:55:40:82:44
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0×3): frags=normal miss_send_len=0
ovs-dpctl show
system@ovs-system:
lookups: hit:2794001 missed:30 lost:0
flows: 0
port 0: ovs-system (internal)
port 1: ovsbr (internal)
port 2: vx1 (vxlan: remote_ip=15.0.113.3)
ovs-vsctl show
6a1f48dd-30d8-4d1b-b52a-4698354e5c26
Bridge ovsbr
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”15.0.113.3″}
Port ovsbr
Interface ovsbr
type: internal
Here’s the modinfo for the openvswitch module:
Server 1) modinfo openvswitch
filename: /lib/modules/3.5.0-030500-generic/kernel/net/openvswitch/openvswitch.ko
version: 1.9.90
license: GPL
description: Open vSwitch switching datapath
srcversion: 726FDD22BBD59C95CB6769A
depends:
vermagic: 3.5.0-030500-generic SMP mod_unload modversions
Server 2) modinfo openvswitch
filename: /lib/modules/3.5.0-17-generic/kernel/net/openvswitch/openvswitch.ko
version: 1.9.90
license: GPL
description: Open vSwitch switching datapath
srcversion: 726FDD22BBD59C95CB6769A
depends:
vermagic: 3.5.0-17-generic SMP mod_unload modversions
Please let me know if there is any other debug info I can provide.
Thanks
Radhika
Thanks Radhika, I am in between x86 for a couple weeks but will try and get a loaner or two and try and test myself also.
Thanks,
-Brent
Hi Brent,
Thanks a lot for this tutorial (and others on openflow). I think I have the setup fine, but the bridge that has the gre interface is sending out ARP requests for the remote tunnel IP instead of tunneling.
Can you please help me with the setup here.
I am trying to get a gre tunnel working by using 2 VMs that have openvswitch and 2 VMs as hosts. Following is my setup:
——————- =========
(interface eth1)Switch 2 (interface eth2)
Here is my configuration:
Host 1: The interface connected to the switch-1 has the IP 192.168.5.100
Switch 1:
eth1 is part of a bridge br1 and has IP 192.168.1.1
eth2 is part of another bridge br2 and has IP 192.168.5.1
There is another bridge br3, which has the IP 10.1.1.1 and has a GRE interface with remote IP 192.168.1.100 (which is the ‘public IP’ of switch-2).
Host 2: The interface connected to the switch has the IP 192.168.10.100
Switch 2:
eth1 is part of a bridge br1 and has IP 192.168.1.1008.eth2 is part of another bridge br2 and has IP 192.168.10.1
There is another bridge br3, which has the IP 11.1.1.1 and has a GRE port with remote IP 192.168.1.1 (‘public’ IP of Switch-1)
I have added relevant routes: ‘route add -host 192.168.10.100 dev br2′ on Switch-1 and ‘route add -host 192.168.5.1 00 dev br2′ on Switch-1.
Now, when I try to ping from Host 1 to Host 2 using the tunnel, I did a tcpdump on br2 on Switch-1(the bridge having gre interface). br2 is sending out an ARP request for 192.168.10.100 instead of tunneling the packet. Can you please tell me what I may be missing here? Do we have to add flows manually?
Thanks & Regards,
Ramachandra
Hi Brent,
I just observed my setup depiction in the above comment is not clear (a pasting error probably). This is the setup that I am using:
——————- =========
(interface eth1)Switch 2 (interface eth2)
Can you please help me with understanding what I am missing here.
Thanks & Regards,
Ramachandra Kasyap
Again the same issue. I apologize for multiple posts, but the content looks fine in the text box, but somehow does not show up properly in the post. Making a final attempt:
————————- —————connected to—
—–connected to————————-
I am trying to ping from Host 1 to Host 2 over the tunnel.
Thanks,
Kasyap
I see that we don’t need 3 bridges – only 2 bridges would do:
Switch -1:
br1 (which has eth1) and IP 192.168.1.1
br2 (which has eth2 and gre10). br2 has IP 192.168.5.1. Remote IP of gre10 is 192.168.1.100.
There is a VM connected to eth2 with the IP 192.168.5.100
Similarly, Switch-2 has the following configuration:
br1 (which has eth1) and IP 192.168.1.100
br2 (which has eth2 and gre10). br2 has IP 192.168.10.1. Remote IP of gre10 is 192.168.1.1.
There is a VM connected to eth2 with the IP 192.168.10.100
I am trying ping from VM1 (with IP 192.168.5.100) to VM2 (192.168.10.100). But, I still face the same issue – at Switch 1, an ARP request is sent out on br2 for 192.168.10.100. I am not sure what I am missing.
Thanks & Regards,
Ramachandra
Debugging further, I think I know where the issue is (though I am not sure how to fix this).
In openvswitchd.log, I see the following messages when I add the port for gre:
ovs-vsctl add-port br2 gre10 …
WARN|system@br2: failed to add gre10 as port: Address family not supported by protocol.
I did not check this earlier as the ‘ovs-vsctl add-port ..’ command was not throwing any errors. I got suspicious as the gre port was not shown in ‘ovs-dpctl show’, then enabled logging and retried adding bridges and ports again.
If anyone faced this issue and fixed it, please let me know.
Thanks & Regards,
Ramachandra
Hi,
Please help me , I could not ping VM from one host to another host via VXLAN
I have installed OVS 1.7.3 in 2 hosts , configuration is below
Created 2 VMs in Host 1 and 1 VM in Host 2 , added VMs interface (vnet0, vnet1) in the bridge “br2″ and physical port in “br1″
Host 1:
——–
ovs-vsctl add-br br1
ovs-vsctl add-br br2
ovs-vsctl add-port br1 p6p1
ifconfig p6p1 0
ovs-vsctl add-port br2 vnet0
ovs-vsctl add-port br2 vnet1
ifconfig br1 192.168.1.11 netmask 255.255.255.0
route add default gw 192.168.1.1 br1
ifconfig br2 10.1.2.11 netmask 255.255.255.0
ovs-vsctl add-port br2 vx1 — set interface vx1 type=vxlan options:remote_ip=192.168.1.10
VM1 IP : 10.1.2.50
VM2 IP : 10.1.2.51
Host 2 :
——–
ovs-vsctl add-br br1
ovs-vsctl add-br br2
ovs-vsctl add-port br1 p20p1
ifconfig p20p1 0
ovs-vsctl add-port br2 vnet0
ifconfig br1 192.168.1.10 netmask 255.255.255.0
route add default gw 192.168.1.1 br1
ifconfig br2 10.1.2.10 netmask 255.255.255.0
ovs-vsctl add-port br2 vx1 — set interface vx1 type=vxlan options:remote_ip=192.168.1.11
VM1 IP : 10.1.2.52
Any idea?
Hi Brent,
GRE is working fine in this scenario, could ping from VM to VM by GRE, it is encapsulating with GRE header.
But in VXLAN it is not working
br1 ping is successfull from both the host.
Host 1 :
======
[root@VERYX home]# ping 192.168.1.10
PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.
64 bytes from 192.168.1.10: icmp_req=1 ttl=64 time=0.542 ms
64 bytes from 192.168.1.10: icmp_req=2 ttl=64 time=0.213 ms
ovs-vsctl show
——————-
6924d99b-c848-4b24-a7ca-f837647cd9ff
Bridge “br2″
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”192.168.1.10″}
Port “br2″
Interface “br2″
type: internal
Port “vnet0″
Interface “vnet0″
Bridge “br1″
Port “br1″
Interface “br1″
type: internal
Port “p6p1″
Interface “p6p1″
route -n
———–
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.66.129 0.0.0.0 UG 0 0 0 p5p1
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.12.0 192.168.66.129 255.255.255.0 UG 0 0 0 p5p1
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 p5p1
br1: flags=4163 mtu 9000
inet 192.168.1.11 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::58d1:f6ff:fed5:2942 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:85 txqueuelen 0 (Ethernet)
RX packets 266 bytes 37372 (36.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 245 bytes 23359 (22.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br2: flags=4163 mtu 1500
inet 10.1.2.11 netmask 255.255.255.0 broadcast 10.1.2.255
inet6 fe80::5446:bbff:fe8e:2e4a prefixlen 64 scopeid 0×20
ether 56:46:bb:8e:2e:4a txqueuelen 0 (Ethernet)
RX packets 19890 bytes 35230671 (33.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 22500 bytes 3325783 (3.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 16436
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0×10
loop txqueuelen 0 (Local Loopback)
RX packets 6144579 bytes 9900599294 (9.2 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6144579 bytes 9900599294 (9.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
p5p1: flags=4163 mtu 1500
inet 192.168.66.130 netmask 255.255.255.0 broadcast 192.168.66.255
inet6 fe80::230:48ff:feba:3b84 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:84 txqueuelen 1000 (Ethernet)
RX packets 12188226 bytes 10304812103 (9.5 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16014534 bytes 16100938795 (14.9 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0200000-d0220000
p6p1: flags=4419 mtu 9000
inet6 fe80::230:48ff:feba:3b85 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:85 txqueuelen 1000 (Ethernet)
RX packets 216483 bytes 183723740 (175.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 88504 bytes 6230355 (5.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0300000-d0320000
vnet0: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe49:b4da prefixlen 64 scopeid 0×20
ether fe:54:00:49:b4:da txqueuelen 500 (Ethernet)
RX packets 119043 bytes 186635778 (177.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 246158 bytes 185881453 (177.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vnet1: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe7b:e96d prefixlen 64 scopeid 0×20
ether fe:54:00:7b:e9:6d txqueuelen 500 (Ethernet)
RX packets 7570 bytes 614957 (600.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11730 bytes 146176530 (139.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Host 2:
=====
[root@VXLAN home]# ping 192.168.1.11
PING 192.168.1.11 (192.168.1.11) 56(84) bytes of data.
64 bytes from 192.168.1.11: icmp_req=1 ttl=64 time=0.526 ms
64 bytes from 192.168.1.11: icmp_req=2 ttl=64 time=0.203 ms
ovs-vsctl show
——————
573e0df5-1c70-420f-b31a-8ddf3e4ca5da
Bridge “br1″
Port “p20p1″
Interface “p20p1″
Port “br1″
Interface “br1″
type: internal
Bridge “br2″
Port “vnet0″
Interface “vnet0″
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”192.168.1.11″}
Port “br2″
Interface “br2″
type: internal
route -n
———-
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.66.129 0.0.0.0 UG 0 0 0 p21p1
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 p21p1
ifconfig -a
————-
br1: flags=4163 mtu 1500
inet 192.168.1.10 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::260:e0ff:fe49:455e prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5e txqueuelen 0 (Ethernet)
RX packets 48 bytes 6678 (6.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 56 bytes 8854 (8.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br2: flags=4163 mtu 1500
inet 10.1.2.10 netmask 255.255.255.0 broadcast 10.1.2.255
inet6 fe80::1c78:87ff:fe14:9740 prefixlen 64 scopeid 0×20
ether 1e:78:87:14:97:40 txqueuelen 0 (Ethernet)
RX packets 34 bytes 4910 (4.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 7066 (6.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 16436
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0×10
loop txqueuelen 0 (Local Loopback)
RX packets 104165 bytes 199828052 (190.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 104165 bytes 199828052 (190.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
p16p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5a txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0100000-d0120000
p17p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5b txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0200000-d0220000
p18p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5c txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 18 memory 0xd0300000-d0320000
p19p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 19 memory 0xd0400000-d0420000
p20p1: flags=4163 mtu 1500
inet6 fe80::260:e0ff:fe49:455e prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5e txqueuelen 1000 (Ethernet)
RX packets 49 bytes 6934 (6.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 60 bytes 9696 (9.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0500000-d0520000
p21p1: flags=4163 mtu 1500
inet 192.168.66.131 netmask 255.255.255.0 broadcast 192.168.66.255
inet6 fe80::260:e0ff:fe49:455f prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5f txqueuelen 1000 (Ethernet)
RX packets 93124 bytes 8012610 (7.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 156267 bytes 194696815 (185.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0600000-d0620000
p33p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:59 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0000000-d0020000
vnet0: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe9e:cb75 prefixlen 64 scopeid 0×20
ether fe:54:00:9e:cb:75 txqueuelen 500 (Ethernet)
RX packets 243 bytes 24945 (24.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3611 bytes 192533 (188.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Thanks & Regards,
Arunkumar U
Hey Arunkumar, paste in the following for me if still having problems from the two physical hosts.
route -n
ifconfig -a
ovs-vsctl show
Ensure the two br1 interfaces can ping one another. e.g, ping 192.168.1.10
Thanks,
-Brent
Hi Brent,
GRE is working fine in this scenario, could ping from VM to VM by GRE, it is encapsulating with GRE header.
But in VXLAN it is not working
br1 ping is successfull from both the host.
Host 1 :
======
[root@VERYX home]# ping 192.168.1.10
PING 192.168.1.10 (192.168.1.10) 56(84) bytes of data.
64 bytes from 192.168.1.10: icmp_req=1 ttl=64 time=0.542 ms
64 bytes from 192.168.1.10: icmp_req=2 ttl=64 time=0.213 ms
ovs-vsctl show
——————-
6924d99b-c848-4b24-a7ca-f837647cd9ff
Bridge “br2″
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”192.168.1.10″}
Port “br2″
Interface “br2″
type: internal
Port “vnet0″
Interface “vnet0″
Bridge “br1″
Port “br1″
Interface “br1″
type: internal
Port “p6p1″
Interface “p6p1″
route -n
———–
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.66.129 0.0.0.0 UG 0 0 0 p5p1
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.12.0 192.168.66.129 255.255.255.0 UG 0 0 0 p5p1
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 p5p1
br1: flags=4163 mtu 9000
inet 192.168.1.11 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::58d1:f6ff:fed5:2942 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:85 txqueuelen 0 (Ethernet)
RX packets 266 bytes 37372 (36.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 245 bytes 23359 (22.8 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br2: flags=4163 mtu 1500
inet 10.1.2.11 netmask 255.255.255.0 broadcast 10.1.2.255
inet6 fe80::5446:bbff:fe8e:2e4a prefixlen 64 scopeid 0×20
ether 56:46:bb:8e:2e:4a txqueuelen 0 (Ethernet)
RX packets 19890 bytes 35230671 (33.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 22500 bytes 3325783 (3.1 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 16436
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0×10
loop txqueuelen 0 (Local Loopback)
RX packets 6144579 bytes 9900599294 (9.2 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6144579 bytes 9900599294 (9.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
p5p1: flags=4163 mtu 1500
inet 192.168.66.130 netmask 255.255.255.0 broadcast 192.168.66.255
inet6 fe80::230:48ff:feba:3b84 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:84 txqueuelen 1000 (Ethernet)
RX packets 12188226 bytes 10304812103 (9.5 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16014534 bytes 16100938795 (14.9 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0200000-d0220000
p6p1: flags=4419 mtu 9000
inet6 fe80::230:48ff:feba:3b85 prefixlen 64 scopeid 0×20
ether 00:30:48:ba:3b:85 txqueuelen 1000 (Ethernet)
RX packets 216483 bytes 183723740 (175.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 88504 bytes 6230355 (5.9 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0300000-d0320000
vnet0: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe49:b4da prefixlen 64 scopeid 0×20
ether fe:54:00:49:b4:da txqueuelen 500 (Ethernet)
RX packets 119043 bytes 186635778 (177.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 246158 bytes 185881453 (177.2 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vnet1: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe7b:e96d prefixlen 64 scopeid 0×20
ether fe:54:00:7b:e9:6d txqueuelen 500 (Ethernet)
RX packets 7570 bytes 614957 (600.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 11730 bytes 146176530 (139.4 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Host 2:
=====
[root@VXLAN home]# ping 192.168.1.11
PING 192.168.1.11 (192.168.1.11) 56(84) bytes of data.
64 bytes from 192.168.1.11: icmp_req=1 ttl=64 time=0.526 ms
64 bytes from 192.168.1.11: icmp_req=2 ttl=64 time=0.203 ms
ovs-vsctl show
——————
573e0df5-1c70-420f-b31a-8ddf3e4ca5da
Bridge “br1″
Port “p20p1″
Interface “p20p1″
Port “br1″
Interface “br1″
type: internal
Bridge “br2″
Port “vnet0″
Interface “vnet0″
Port “vx1″
Interface “vx1″
type: vxlan
options: {remote_ip=”192.168.1.11″}
Port “br2″
Interface “br2″
type: internal
route -n
———-
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.66.129 0.0.0.0 UG 0 0 0 p21p1
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br2
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.66.0 0.0.0.0 255.255.255.0 U 0 0 0 p21p1
ifconfig -a
————-
br1: flags=4163 mtu 1500
inet 192.168.1.10 netmask 255.255.255.0 broadcast 192.168.1.255
inet6 fe80::260:e0ff:fe49:455e prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5e txqueuelen 0 (Ethernet)
RX packets 48 bytes 6678 (6.5 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 56 bytes 8854 (8.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
br2: flags=4163 mtu 1500
inet 10.1.2.10 netmask 255.255.255.0 broadcast 10.1.2.255
inet6 fe80::1c78:87ff:fe14:9740 prefixlen 64 scopeid 0×20
ether 1e:78:87:14:97:40 txqueuelen 0 (Ethernet)
RX packets 34 bytes 4910 (4.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 35 bytes 7066 (6.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 16436
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0×10
loop txqueuelen 0 (Local Loopback)
RX packets 104165 bytes 199828052 (190.5 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 104165 bytes 199828052 (190.5 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
p16p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5a txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0100000-d0120000
p17p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5b txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0200000-d0220000
p18p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5c txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 18 memory 0xd0300000-d0320000
p19p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:5d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 19 memory 0xd0400000-d0420000
p20p1: flags=4163 mtu 1500
inet6 fe80::260:e0ff:fe49:455e prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5e txqueuelen 1000 (Ethernet)
RX packets 49 bytes 6934 (6.7 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 60 bytes 9696 (9.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0500000-d0520000
p21p1: flags=4163 mtu 1500
inet 192.168.66.131 netmask 255.255.255.0 broadcast 192.168.66.255
inet6 fe80::260:e0ff:fe49:455f prefixlen 64 scopeid 0×20
ether 00:60:e0:49:45:5f txqueuelen 1000 (Ethernet)
RX packets 93124 bytes 8012610 (7.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 156267 bytes 194696815 (185.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17 memory 0xd0600000-d0620000
p33p1: flags=4099 mtu 1500
ether 00:60:e0:49:45:59 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 memory 0xd0000000-d0020000
vnet0: flags=4163 mtu 1500
inet6 fe80::fc54:ff:fe9e:cb75 prefixlen 64 scopeid 0×20
ether fe:54:00:9e:cb:75 txqueuelen 500 (Ethernet)
RX packets 243 bytes 24945 (24.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3611 bytes 192533 (188.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Thanks & Regards,
Arunkumar U
Hi Brent,
VXLAN is not working in the above scenario, do you have any idea ?
Thanks & Regards,
Arunkumar U
Is the BRCOMPAT taken out ? After I do ‘make && make install’. The openswitch module is there but the BRCOMPAT module is missing.
Is the brcompat module still required to run with the openswitch ?
Thanks,
Roni
Hi Roni, It is no longer recommended after the upstream in 3.2. I removed it, thanks for the feedback. I will verify the build, real quick.
I verified the build. I changed br2 -> br1 and br1-> br0 as I think it confused folks looking for br0 to equal eth0. The 2nd bridge , br1 is just to setup bring up an interface. That would normally be a VM. You can build the tunnels lots of ways. I use a second br since the hosts are both on the same subnet in my laptop on VM Fusion, so I need to have seperate networks on br1 on the backside that arent reachable. Here was the output from one of the hosts with config after OVS was built.
root@ub64:/home/brent/openvswitch# ovs-vsctl add-br br0
root@ub64:/home/brent/openvswitch# ovs-vsctl add-br br1
root@ub64:/home/brent/openvswitch# ovs-vsctl add-port br0 eth0
root@ub64:/home/brent/openvswitch# ifconfig eth0 0 && ifconfig br0 192.168.1.10 netmask 255.255.255.0
root@ub64:/home/brent/openvswitch# route add default gw 192.168.1.1 br0
root@ub64:/home/brent/openvswitch# ifconfig br1 10.1.2.10 netmask 255.255.255.0
root@ub64:/home/brent/openvswitch# ovs-vsctl add-port br1 gre1 — set interface gre1 type=gre options:remote_ip=192.168.1.11
root@ub64:/home/brent/openvswitch#
root@ub64:/home/brent/openvswitch# ping 10.1.2.11
PING 10.1.2.11 (10.1.2.11) 56(84) bytes of data.
64 bytes from 10.1.2.11: icmp_req=1 ttl=64 time=0.625 ms
64 bytes from 10.1.2.11: icmp_req=2 ttl=64 time=0.313 ms
^C
— 10.1.2.11 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.313/0.469/0.625/0.156 ms
root@ub64:/home/brent/openvswitch# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 br0
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
root@ub64:/home/brent/openvswitch# ifconfig
br0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82
inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::285f:3ff:fea2:dafc/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:622 errors:0 dropped:0 overruns:0 frame:0
TX packets:54 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:171756 (171.7 KB) TX bytes:3832 (3.8 KB)
br1 Link encap:Ethernet HWaddr ea:34:7d:49:e7:49
inet addr:10.1.2.10 Bcast:10.1.2.255 Mask:255.255.255.0
inet6 addr: fe80::c0a7:aaff:fe90:29dc/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:896 (896.0 B) TX bytes:986 (986.0 B)
eth0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82
inet6 addr: fe80::20c:29ff:fe65:fd82/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1222493 errors:0 dropped:0 overruns:0 frame:0
TX packets:53824 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:653399936 (653.3 MB) TX bytes:4089412 (4.0 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:7088 errors:0 dropped:0 overruns:0 frame:0
TX packets:7088 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:537609 (537.6 KB) TX bytes:537609 (537.6 KB)
root@ub64:/home/brent/openvswitch# ps -ea | grep ovs
27288 ? 00:00:00 ovs_workq
27293 ? 00:00:00 ovsdb-server
27296 ? 00:00:00 ovs-vswitchd
27297 ? 00:00:00 ovs-vswitchd
root@ub64:/home/brent/openvswitch# ovs-vsctl del-port gre1
root@ub64:/home/brent/openvswitch# ovs-vsctl add-port br1 vx1 — set interface vx1 type=vxlan options:remote_ip=192.168.1.11
root@ub64:/home/brent/openvswitch# ping 10.1.2.11
PING 10.1.2.11 (10.1.2.11) 56(84) bytes of data.
64 bytes from 10.1.2.11: icmp_req=1 ttl=64 time=0.810 ms
^C
— 10.1.2.11 ping statistics —
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.810/0.810/0.810/0.000 ms
Thanks,
-Brent
Brent,
great info – it is awesome to find clear and informative details.
Has there been any update on the mulicast support and VXLAN?
Thanks, Mas