published: 23rd of November 2017
Libvirt has the ability to create a pseudo-wire between virtual guest interfaces using either TCP or UDP. The advantage of using pseud-wires is that you do not need to create virtual switches to attach your guest VM interfaces to.
Virtualbox has an annoying trait of stripping vlan tags on interfaces which are not of the type virtio . Since some network device Vagrant boxes do not support virtio interfaces you are not able to test alot of switching related technologies.
This post will cover how to define a UDP connection between nodes in a Vagrantfile . For reference the following software will be used in this post.
The following topology will be created as part of this lab.
Tunnels are created by mapping IPv4 addresses and port numbers to create a logical endpoint that is tied to a switchport. A little planning never goes astray, especially when you are building larger topologies. This table of configuration items describes our point to point connections between hosts.
Hostname | MAC Address | Local VM Loopback IP | Local Tunnel Port | Remote VM Loopback IP | Remote Tunnel Port | Interface Description |
---|---|---|---|---|---|---|
spine01 | 28:b7:ad:00:00:01 | 127.1.1.1 | 10001 | 127.1.1.2 | 10001 | swp1 |
spine02 | 28:b7:ad:00:00:02 | 127.1.1.2 | 10001 | 127.1.1.1 | 10001 | swp1 |
Tunnel interfaces are defined in a Vagranfile under the network type private_network. There are a number of configuration parameters that need to be configured, let go through them.
Use this Vagrantfile to build the above topology.
# -*- mode: ruby -*-
# vi: set ft=ruby :
$script = <<SCRIPT
sudo net add hostname $1
sudo net add int swp1 ip address $2
sudo net commit
SCRIPT
Vagrant.configure("2") do |config|
##########################
# spine01 #
##########################
config.vm.define "spine01" do |node|
node.vm.hostname = "spine01"
node.vm.box = "CumulusCommunity/cumulus-vx"
node.vm.synced_folder ".", "/vagrant", disabled: true
node.ssh.insert_key = false
# Provider-specific configuration
node.vm.provider :libvirt do |domain|
domain.nic_adapter_count = 52
end
node.vm.network :private_network,
:mac => "28:b7:ad:00:00:01",
:libvirt__tunnel_type => "udp",
:libvirt__tunnel_local_ip => "127.1.1.1",
:libvirt__tunnel_local_port => "10001",
:libvirt__tunnel_ip => "127.1.1.2",
:libvirt__tunnel_port => "10001",
:libvirt__iface_name => "swp1",
auto_config: false
node.vm.provision "shell", inline: $script, :args => ["spine01", "10.1.1.1/30"]
end
##########################
# spine02 #
##########################
config.vm.define "spine02" do |node|
node.vm.box = "CumulusCommunity/cumulus-vx"
node.vm.synced_folder ".", "/vagrant", disabled: true
node.vm.hostname = "spine02"
node.ssh.insert_key = false
# Provider-specific configuration
node.vm.provider :libvirt do |domain|
domain.nic_adapter_count = 52
end
node.vm.network :private_network,
:mac => "28:b7:ad:00:00:02",
:libvirt__tunnel_type => "udp",
:libvirt__tunnel_local_ip => "127.1.1.2",
:libvirt__tunnel_local_port => "10001",
:libvirt__tunnel_ip => "127.1.1.1",
:libvirt__tunnel_port => "10001",
:libvirt__iface_name => "swp1",
auto_config: false
node.vm.provision "shell", inline: $script, :args => ["spine02", "10.1.1.2/30"]
end
end
Ok vagrant up and ssh into spine01 so we can test that the interface has connectivity between spine01 and spine02 over the swp1 interfaces we created.
First lest check the interface is up and has an IP address.
sudo net show int
# output
Name Master Speed MTU Mode Remote Host Remote Port Summary
-- ------ -------- ------- ----- ------------ ------------- ------------- ---------------------------
UP lo None N/A 65536 Loopback IP: 127.0.0.1/8, ::1/128
UP eth0 None 1G 1500 Mgmt IP: 192.168.121.68/24(DHCP)
UP swp1 None 1G 1500 Interface/L3 cumulus swp1 IP: 10.1.1.1/30
Great the interface is up, now lets test that we can ping the other side.
ping 10.1.1.2 -c 2
# output
PING 10.1.1.2 (10.1.1.2) 56(84) bytes of data.
64 bytes from 10.1.1.2: icmp_seq=1 ttl=64 time=0.365 ms
64 bytes from 10.1.1.2: icmp_seq=2 ttl=64 time=0.480 ms
--- 10.1.1.2 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.365/0.422/0.480/0.061 ms
vagrant@spine01:~$
Awesome, we can ping, as a final check, lets confirm the MAC address that was configured on the other side is learned via ARP.
ip neighbor
# output
192.168.121.1 dev eth0 lladdr fe:54:00:6e:2e:bd REACHABLE
10.1.1.2 dev swp1 lladdr 28:b7:ad:00:00:02 DELAY
fe80::fc54:ff:fe6e:2ebd dev eth0 lladdr fe:54:00:6e:2e:bd STALE
vagrant@spine01:~$
Good stuff we are learning the correct MAC address over the UDP tunnel. Dont forget to vagrant destroy -f to clean up the lab.
Using UDP tunneling to create pseudo-wires between host interfaces in Vagrant is a great way to get around having to build multiple virtual switches. It also has the added advantage of passing all traffic between host interfaces and not stripping out things like VLAN tags.