GoBGP – A Control Plane Evolving Software Networking
If I have talked to you over the past year or two about networking, odds are I have mentioned a project to you called GoBGP. If we have spoken a lot, you are probably sick of me talking to you about GoBGP 🙂 There is a reason, it’s because GoBGP is a missing link in networking that has been for a long time. GoBGP is a project from some really smart and just as importantly friendly, community minded folks at NTT Communications (largest telco in Japan). The software group that does the software is OSRG and the same organization that did RYU the SDN controller that was popular in OpenStack deployments. GoBGP is a BGP focused routing daemon that is incredibly flexible, written in Go and perfect for those both looking for practical control plane SDN solutions or network engineers looking to figure out what the hell their future will look like and where to start evolving in a software driven age. The tutorial will also instruct you how to install the Go programming language and environment in order to pull down and build the GoBGP binaries.
There are a few software router implementations, all useful in their own right. The first reason I prefer GoBGP is its written in Go, which gives you many of the benefits of C with a lower barrier to entry then C along with great concurrency support along with great native tooling. More on reasons to use Go here. GoBGP is also completely cross platform and does not have any hardware specific dependencies. Secondly, its architecture is outstanding and very modular unlike the monolithic alternatives. As an example, they implemented a draft I am huge fan of from Petr Lapukhov of Facebook named BGP for opaque signaling in a couple of days without needing to change anything internally. I see this level of modularity at work and recognize the level of experience required to design such nice architectures like the OSRG team have put together to seamlessly support EVPN, Opaque Flowspec, RRs, RPKI, BMP to name a few.
Download and Install GoBGP
The team does a great job with documentation so anything I write will duplicitous but I will dump my quick notes to get up and running. There is not a binary download for Mac/Linux/Windows explicitly so its a great chance for you to learn something new by installing the Go compiler if you haven’t before. Its fast and easy. If on a Mac or Linux box it is the same.
Pre-Requisites:
- Windows or Mac there are installers found at Go Downloads.
- Also Install git if you want to compile from source at the end of the post.
To install on Linux, paste the following or follow these instructions:
|
1 2 3 4 5 6 7 8 |
cd /usr/local sudo wget https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz # Untar into /usr/local/go That is the generally excepted location. sudo tar xvzf go1.6.linux-amd64.tar.gz # Add the go bin directory to your executable path. export PATH=$PATH:/usr/local/go/bin |
The next part of a Go install is the part that trips people up the most. Go uses a file structure for libraries that has to follow a convention because libraries are statically linked. The rational for this is because the compile time is incredibly fast because there is not any dynamic linking that eats up compile time searching for libraries.
Wherever you are going to store your Go project need to be contained in the exported to the $GOPATH directory. The docs use this example:
|
1 2 3 |
export GOPATH=$HOME/work |
I store all of my Go projects in a directory in my home directory named, you guessed it, go. Im going to spell out full file paths to keep it as simple as possible. You want to add the export GOPATH=
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# Set the Mac OS X GOPATH with: $ export GOPATH=/Users/brent/code/go # Now your env should contain that path $ env | grep GO GOPATH=/Users/brent/code/go # Echo it to your .bashrc so it will be persistent $ echo GOPATH=/Users/brent/code/go >> ~/.bash_profile # Reread your bashrc and verify with `env` again $ source ~/.bash_profile |
On a Linux box I use /home/brent/go:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
# My Linux Hosts $ export GOPATH=/home/brent/go # Verify the Linux exports with $ env | grep GO GOPATH=/home/brent/go # Echo it to your .bashrc so it will be persistent $ echo GOPATH=/home/brent/go >> ~/.bashrc # Reread your bashrc and verify with `env` again $ source ~/.bashrc |
All of your libraries are going to be installed into the $GOPATH directory path.
Now you should be ready to download GoBGP:
|
1 2 3 4 |
$ go get github.com/osrg/gobgp/gobgpd $ go get github.com/osrg/gobgp/gobgp |
There are two binaries that should be in your file path now, gobgp (client cli) and gobgpd (daemon).
|
1 2 3 4 |
gobgp --help gobgpd --help |
GoBGP Download and Run
Now run gobgp with an example config. The following toml formatted config points to another BGP instance both using AS: 65000. The source IP is 192.168.1.103 and the remote BGP peer is 192.168.1.249. Simply create a text file containing that config with your IP address swapped in.
|
1 2 3 4 5 6 7 8 9 10 |
[global.config] as = 65000 router-id = "192.168.1.103" [[neighbors]] [neighbors.config] neighbor-address = "192.168.1.249" peer-as = 65000 |
Now start the daemon using the config file I saved as gobgp-conf-151.toml and using -l debug for debug logging so you can see the sausage factory go:
|
1 2 3 4 |
# $ gobgpd -l debug --config-file=gobgp-conf-151.toml |
Assuming you have a router, another gobgp, quagga etc looking to peer with the the host you start gobgpd on view the neighbor adjacency with:
|
1 2 3 4 5 |
# gobgp neighbor Peer AS Up/Down State |#Advertised Received Accepted 192.168.1.249 65000 07:27:46 Establ | 2 1 1 |
From there, the options are massive. Some of the highlights for example is, I want to monitor a route and do something when the route appears. It is super easy to integrate code with this project. A simple example is something I did just reading CLI output here to monitor when a container prefix appears to plumb routes. Note, that project currently is defunct but we have some cool stuff coming up.
GoBGP Example Commands
Here is my CLI cheatsheet. The OSRG folks have amazing docs so please use those but figured I would dump them here anyways to help you get an overview of how versatile this project is.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
############> Neighbor Options <############ # Disable a BGP neighbor gobgp neighbor 192.168.1.248 disable # Enable a BGP neighbor gobgp neighbor 192.168.1.248 enable # List BGP neighbors gobgp neighbor ############> Custom Neighbor Ops <############ gobgp neighbor 192.168.1.248 adj-in gobgp neighbor 192.168.1.248 adj-out gobgp neighbor 192.168.1.248 local -a ipv4 #################> BGP Neighbor Resets <############### gobgp neighbor 192.168.1.248 disable gobgp neighbor 192.168.1.248 enable gobgp neighbor 192.168.1.248 softreset gobgp neighbor 192.168.1.248 softresetin gobgp neighbor 192.168.1.248 softresetout gobgp neighbor 192.168.1.248 reset ############> Extended Community <############ # Add Policy gobgp policy extcommunity add ecs1 RT:65100:10 # Del Policy gobgp policy neighbor del ecs1 ############> Add/Del VRF <############ # Add a VRF gobgp vrf add <vrf name> rd <rd> rt {import|export|both} <rt>... # Del a VRF gobgp vrf del <vrf name> # Show VRFs gobgp vrf ##########> Add Del Routes to a VRF <############ # Add routes to a VRF gobgp vrf <vrf name> rib add <prefix> -a <address family> # Del routes from a VRF gobgp vrf <vrf name> rib del <prefix> -a <address family> # Show routes in vrf gobgp vrf <vrf name> # Example: gobgp vrf FOO rib add 251.1.1.10/32 -a ipv4 ##############> Global RIB Ops <################ # Add a global rib route gobgp global rib add <prefix> [-a <address family>] # Delete a specific Route gobgp global rib del <prefix> [-a <address family>] # show all Route information gobgp global rib [-a <address family>] # show a specific route information gobgp global rib [<prefix>|<host>] [-a <address family>] # Del v6 RIB entry gobgp global rib del 2001:123:123:1::/64 -a ipv6 # Add v4 RIB entry gobgp global rib add 251.1.1.100/32 -a ipv4 gobgp global rib add <prefix> [-a <address family>] gobgp global rib del <prefix> [-a <address family>] gobgp global rib [-a <address family>] gobgp global rib [<prefix>|<host>] [-a <address family>] gobgp global rib del 2001:123:123:1::/64 -a ipv6 gobgp global rib add 251.1.1.100/32 -a ipv4 ################> Monitor Neighbors <############# # monitor neighbor status gobgp monitor neighbor # monitor specific neighbor status gobgp monitor neighbor <neighbor address> #############> Monitor Rib Entries <############# gobgp monitor global rib # Example Output # [ROUTE] 10.0.0.0/24 via 0.0.0.0 aspath [] attrs [{Origin: i}] # monitor command blocks. add routes from another terminal gobgp global rib add 10.0.0.0/24 # monitor global rib changes gobgp monitor global rib |
GoBGP Build from Source
For those new to Go, I figured I could add a bit of value for the OSRG project by explaining a bit more on pulling the Go source code and building a binary from them. The binaries for Go are stored in $GOPATH/bin which in my case is /home/brent/go/bin on a Linux OS. If you want to pull the GoBGP source code and build it locally do the following.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
# cd $GOPATH is the same as cd /home/brent/go for me $ cd $GOPATH # Create a src/github directory so the source files are stored in a static location the compiler knows where to look for. $ mkdir -p src/github.com $ cd src/github.com # or $ cd $GOPATH/src/github.com # or :) $ cd /home/<username>/go/src/github.com # Clone the github.com/osrg/gobgp source repo $ git clone https://github.com/osrg/gobgp.git $ cd osrg/gobgp # Now we need to pull down the libraries that are used by gobgp. # Go makes this very easy. go get ./... will recursively look for # any dependencies below /home/<username>/go/src/github.com/osrg/gobgp $ go get ./.. # Then build the src code using or install the binary using go install ./... $ go build ./... # Now you will see binaries in the $GOPATH/bin/ directory ls $GOPATH/bin/ |
More Resources
This doesn’t even scratch the surface. There is plenty of valid reasons where a control plane belongs. L3 obviously scales further and is more reliable then large L2 domains. If you are looking to push routing closer to the edge, eventually you will consider where the L3 control belongs. Containers using Ipvlan L3 will require routes to be injected into the network. That could be a vendor HW integration installing routes or it could be the edge host signaling new routes. Either way, instead of announcing MAC addresses into a ToR, its just as reasonable to expect folks to want to announce routes into ToRs. If you want to eliminate L2, it requires pushing L3 further to the edge where the density is and will rapidly grow.
Thanks to the OSRG folks for being such good stewards in open source and the support from their parent company NTT in supporting them. Explore the github.com/osrg/gobgp project, there is a lot to learn there. The APIs for development are cutting edge with tons of opportunity for users, devs, vendors and those looking to evolve their skills in both NetOps and development, or both. They have a massive BGP network, they know it well, there is a lot everyone of all levels can learn there. Pretty reasonable to start expecting containerized control planes to bolt on top of any datapath sooner rather then later so why not get ahead of the curve. If you have any troubles use the project’s repo issues to get assistance as I am regrettably slow replying to comments here as time is not my friend atm 🙂
Hey Brent, great post! It just looks like you’re so excited you’ve forgot to mention what I’ve just mentioned here 🙂
Check out some client code examples, here and here.
Awesome! Thanks Tamihiro! Keep up the great work. I totally missed the PPN post. Thanks!
I wonder how it compares to ExaBGP ? Which is the “go to” ATM.
-Alex
Hey Alex! both exa and gobgp support flowspec. There are some perf numbers I remember from an issue below:
Perf numbers Gobgp v. Quagga and full table load time with gobgp.
https://github.com/osrg/gobgp/issues/323
A toolkit to test:
https://github.com/osrg/bgperf
-More links I should have added I just noticed:
https://ripe71.ripe.net/presentations/135-RIPE71_GoBGP.pdf
-I thought this was fun too:
https://github.com/Exa-Networks/exabgp/wiki/Other-OSS-BGP-implementations
– On different language preference, the gobgp team chose GRPC as an option for APIs which inherently provides ten different language options C, C++, Java, Go, Node.js, Python, Ruby, Objective-C, PHP and C#.
http://www.grpc.io
Cheers!
for those now to > for those new to
Thanks Joop! Busted writing once, reading never 🙂