Ansible, Cumulus, Dell & Nutanix: One big happy family :) – Part2

The road so far…

In Part1, we covered a few things.

  • Design goals etc,
  • Where this was born out of
  • The demo itself.
  • Getting KVM running on a Dell OpenNetworking switch
  • Starting Foundation VM on the switch. (kinda the cool part, rly..)

In tonights episode.

May acts like a tiger, Richard says ‘meow‘ and the I get down with Ansible…. So now I’m depressed I’m not powersliding supercars… but lets move on shall we?

First we’ll dive into the fabric provisioning playbooks showing you a bit more detail.

Phase1(b): Fabric provisioning w/ Ansible

In part1, we started by showing the template files that we primarily modified. Remember this demo was a fork from a ospfunnum-ansible demo.

Lets go a little deeper..

So, the modified ansible demo is pretty simple, lets start by showing the files in play.

cumulus@wbench:~/ansibledemos$ tree
├── ansible.cfg
├── ansible.tree
├── cumulus-linux-ansible-modules
│   ├── library
│   │   ├── cl_img_install
│   │   ├── cl_interface
│   │   ├── cl_license
│   │   ├── cl_ports
│   │   ├── cl_prefix_check
│   │   ├── cl_quagga_ospf
│   │   └── cl_quagga_protocol
│   ├── LICENSE
│   ├── meta
│   │   └── main.yml
│   ├──
│   ├──
│   ├── sample_playbook
|   │   └── #### Snipped to make post readable ###
│   ├── tests
│   │   └── #### Snipped to make post readable ###
│   └──
├── group_vars
│   ├── all
│   └── leaf
├── handlers
│   └── main.yml
├── hosts-ntnx
├── ntnxbasic.yml
└── roles
├── common
│   ├── files
│   │   ├── leaf1.lic
│   │   ├── leaf2.lic
│   │   ├── spine1.lic
│   │   ├── spine2.lic
│   │   └──
│   ├── tasks
│   │   └── main.yml
│   ├── templates
│   │   ├── interfaces.j2
│   │   └── motd.j2
│   └── vars
├── ntnxbasic
│   ├── tasks
│   │   └── main.yml
│   ├── templates
│   │   ├── interfaces.j2
│   │   ├── quagga_config.j2
│   │   └── quagga_daemon.j2
│   └── vars
│      └── main-ntnx.yml
└── portsconfig
├── tasks
│   └── main.yml
└── templates
└── ports_conf_2lt22s.j2
34 directories, 75 files

The files of consequence for this demo are these ones:

File Description
cumulus-ansible-modules/library/* Module library created to do generic cumulus deploy + testing. Using the cl_interface, cl_license and cl_ports modules here.
hosts-ntnx  Site file, describing the lab topology
ntnxbasic.yml The main playbook for this demo, fans out from this point to use most of the other files/modules


License files to enable hardware acceleration.
roles/common/files/  Network cabling diagram in .dot graphing format. Used by Prescriptive Topology Manager
roles/ntnxbasic/tasks/main.yml  Pointer task playbook that calls most of the files.
roles/ntnxbasic/templates/interfaces.j2 Template that expands to form “etc/network/interfaces” file on the switches. References the roles/ntnxbasic/vars/* folder for node-specific variables.
roles/ntnxbasic/var/main-ntnx.yml  Contains node-specific variables that can be looked up and inserted into templates.
roles/portsconfig/tasks/main.yml Task file that builds and applies the etc/cumulus/ports.conf file to the switches to enforce L1 port state.
roles/portsconfig/templates/ports_conf_2lt22s.j2 Template for the etc/cumulus/ports.conf. This file sets each front-panel port’s breakout mode. For example a 40G QSFP port may be either 1x40g or 4x10g.


Pulling apart the playbook

Fundamentally, I’m just standing on the shoulders of giants @lesliegeek, @natmorris and others. Ansible has some great intro documentation to read too.

The ansible playbook is kicked off with the command:

cumulus@wbench:/home/cumulus/ansibledemos$ sudo ansible-playbook -i hosts-ntnx ntnxbasic.yml

The command runs the ntnxbasic.yml playbook against the hosts in the topology file hosts-ntnx. From the ntnxbasic.yml playbook, ansible fans out to the other linked files by defining the hosts within the topology target, roles to apply, location of variables and handler files. See below:

- hosts: leaf
user: root
- common
- portsconfig
- ntnxbasic
- roles/ntnxbasic/vars/main-ntnx.yml
- include : handlers/main.yml

The hosts: masks the type of hosts from the input topology file called in the first command that this playbook should apply to.

I’m going to pick one of the roles, ntnxbasic, to follow through and show how ansible fans out to execute the role. You can then use the same logic to explore the other roles.

Ansible reads the role-name in the playbook shown above, then looks for roles/[role-name]/tasks/main.yml. So lets open up the ntnxbasic role and go from there.

cumulus@wbench:/home/cumulus/ansibledemos$ vi roles/ntnxbasic/tasks/mail.yml


- name: configure /etc/network/interfaces template: src=../templates/interfaces.j2 dest=/etc/network/interfaces
notify: reload networking
#- name: configure quagga daemons file
# template: src=quagga_daemon.j2 dest=/etc/quagga/daemons
#- name: configure quagga configuration
# template: src=quagga_config.j2 dest=/etc/quagga/Quagga.conf
# notify: restart quagga

As you can see, there is one active action in the file; configure /etc/network/interfaces, the other actions are leftover from the ospfunnum demo that we based it on. I left it in place to show how you can additional actions to the role.

The action is pretty simple;

  1. Expand template file- fill out any variables that are referenced in the template including node-specific-info, use roles/ntnxbasic/vars/main-ntnx.yml
  2. Deploy to each node- push the file to /etc/network/interfaces
  3. Notify the system to reload config- if the interface config has been changed, use an event handler that calls ifupdown2 to reload the config.

That brings us back to the interfaces.j2 template and vars/main-ntnx.yml variables files themselves. We showed their contents in part1.
Rather than show the file contents again, let’s jump on leaf1 and show what the outcome of this role is.

cumulus@leaf1$ sudo vi /etc/network/interfaces


# administered by ansible
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5), ifup(8)
# Please see /usr/share/doc/python-ifupdown2/examples/ for examples
## The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet dhcp
#Nutanix host ports
auto swp32s0
iface swp32s0
mstpctl-portadminedge yes
mstpctl-bpduguard yes
auto swp32s1
iface swp32s1
mstpctl-portadminedge yes
mstpctl-bpduguard yes
auto swp32s2
iface swp32s2
mstpctl-portadminedge yes
mstpctl-bpduguard yes
auto swp32s3
iface swp32s3
mstpctl-portadminedge yes
mstpctl-bpduguard yes
#Bonded Inter-Switch-Link (ISL)
auto peerlink
iface peerlink
bond-slaves swp17 swp18
bond-mode 802.3ad
bond-miimon 100
bond-use-carrier 1
bond-lacp-rate 1
bond-min-links 1
bond-xmit-hash-policy layer3+4
#Define the vlan-aware bridge
auto bridge
iface bridge
bridge-vlan-aware yes
bridge-ports glob swp32s0-3 peerlink
bridge-stp on
bridge-vids 100-500
#VLAN interfaces (SVIs)
auto bridge.100
iface bridge.100
address-virtual 44:38:39:ff:00:01

That’s about all there is to it really… the role expands out a template, adds node-specific info, compares the current file on the system, if it’s different it applies the new file and reloads networking… simple :)

Stay tuned for Part3… assuming anyone’s still interested.

Leave a Reply

The opinions expressed on this site are my own and not necessarily those of my employer.

All code, documentation etc is my own work and is licensed under Creative Commons and you are free to use it, at your own risk.

I assume no liability for code posted here, use it at your own risk and always sanity-check it in your environment.