Salt From The Start To The Beginning

14th January 2018

Introduction

As described by their website; Salt is "Event-driven automation for a software-defined world". You gotta love marketing :) Salt is a large project with many features including; configuration management, an event based reactor, cloud management and network automation.

Salt can do ALOT but it does have a relatively steep learning curve. In saying that, Salt does have extensive documentation, although in my experience you have to know how to use Salt a little bit before the documentation starts to make sense.

This post aims to help get you started on your Salt journey and hopefully once you are done you will continue down the Salt mine, it's pretty fun down here !

Architecture

Salt has many components, these are the main ones:

  • Salt Master - Controls minions
  • Salt Minion - Executes code from the master via an agent
  • Salt Proxy Monion - Executes code on hosts that cannot install an agent
  • Message Bus - Messages between Salt master and minions are sent along the message bus

salt-architecture.svg

Lab Environment

In this post I will configure a Centos host as a Salt master and a Cumulus host as a Salt minion.

lab-topology.svg

For reference the code version used in this lab are as follows:

  • Centos - 7 minimal
  • Cumulus - 3.4.3
  • Salt - 17.7.2

Note

I have built this lab with Vagrant boxes so the Firewall and SELinux are already disabled in the Centos image.

Installation

The process of installing Salt and its dependencies differs from system to system, for the most accurate information refer to the specific instructions per platform .

Centos 7

cmd 

sudo yum install https://repo.saltstack.com/yum/redhat/salt-repo-latest-2.el7.noarch.rpm

Update the YUM cache.

cmd 

sudo yum clean expire-cache

Install the Salt master and minion.

cmd 

sudo yum install -y salt-master salt-minion

Note

The salt-minion daemon is also installed on the Salt Master so that we can manage it via salt.

Cumulus

The Cumulus linux 3.X train is based on the Debian Jessie release, therefore the bellow instructions are based on Debian Jessie.

Use vi or nano to add the required repositories to the end of the /etc/apt/sources.list file.

cmd 

# /etc/apt/sources.list
deb http://ftp.us.debian.org/debian/ jessie main contrib non-free
deb http://repo.saltstack.com/apt/debian/8/amd64/latest jessie main

Install the GPG key.

cmd 

wget https://repo.saltstack.com/apt/debian/8/amd64/latest/SALTSTACK-GPG-KEY.pub
sudo apt-key add SALTSTACK-GPG-KEY.pub

Update the APT cache.

cmd 

sudo apt-get update

Install the Salt minion.

cmd 

sudo apt-get install -y salt-minion

Configuration

Salt daemon configuration files use a YAML syntax. If you are not familiar with YAML there is a good overview here

Master

Update the configuration files on the Centos host which is the Salt Master. The /etc/salt/master file specifies the configuration for the salt-master daemon.

Tell the Salt master where the files related to state are located by setting the file_roots parameter in the /etc/salt/master config file.

file 

file_roots:
  base:
    - /srv/salt/
    - /srv/salt/states
    - /srv/salt/templates
    - /srv/salt/files

Pillar data is static variables that relate to a minions and are defined by the Salt administrator. Tell the Salt master where pillar data is located by setting the pillar_roots parameter in the /etc/salt/master config file.

file 

pillar_roots:
  base:
    - /srv/salt/pillar

Create the directories that where just defined above in the /etc/salt/master config file.

cmd 

sudo mkdir -p /srv/salt/{states,templates,files,pillar}

The /etc/salt/minion file specifies the configuration for the salt-minion daemon.

Set the master configuration parameter in the /etc/salt/minion file.

file 

master: 192.168.121.201

Set the id configuration parameter in the /etc/salt/minion file.

file 

id: 192.168.121.201

Restart both the salt-master and salt-minion daemons.

cmd 

sudo systemctl restart salt-master
sudo systemctl restart salt-minion

Enable both the salt-master and salt-minion daemons to start on boot.

cmd 

sudo systemctl enable salt-master
sudo systemctl enable salt-minion

Use systemctl status command to confirm both services are running and enabled.

cmd 

systemctl status salt-master
systemctl status salt-minion

Note

Since I am not running DNS in this lab I am using IP addresses, if DNS is active in the environment you can use host names in lieu of IP addresses.

Minion

The Cumulus host only needs to have its /etc/salt/minion file updated.

Set the master configuration parameter in the /etc/salt/minion file.

file 

master: 192.168.121.201

Set the id configuration parameter in the /etc/salt/minion file.

file 

id: 192.168.121.202

Restart the salt-minion daemon.

cmd 

sudo systemctl restart salt-minion

Enable the salt-minion daemon to start on boot.

cmd 

sudo systemctl enable salt-minion

Confirm the salt-minion service is running and enabled with the systemctl status command.

cmd 

systemctl status salt-minion

Salt Key

Salt uses public key encryption to authenticate hosts. Before a minion can join the message bus its key must be accepted by the Salt Master.

Use the salt-key -L command to view the status of all the minion keys.

cmd 

sudo salt-key -L

# output
Accepted Keys:
Denied Keys:
Unaccepted Keys:
192.168.121.201
192.168.121.202
Rejected Keys:

As you can see there are two unaccepted keys, these belong to both our Centos and Cumulus hosts.

Use the salt-key -A command to accept all the minion keys.

cmd 

sudo salt-key -A --yes

# output
The following keys are going to be accepted:
Unaccepted Keys:
192.168.121.201
192.168.121.202
Key for minion 192.168.121.201 accepted.
Key for minion 192.168.121.202 accepted.

Test Connectivity

Use the salt '*' test.ping command to confirm the Salt master can connect to all the Salt minions. If the master can connect to the minions a True status will be returned.

cmd 

sudo salt '*' test.ping

# output
192.168.121.202:
    True
192.168.121.201:
    True

Salt Command

Performing actions against minions with Salt from the CLI is done with the salt command. The salt command has the following structure.

salt [options] '<target>' <function> [arguments]

  • options - Examples are version and timeout values.
  • target - Used to select minions to execute functions against. '*' targets all minions.
  • function - Action to execute on the minion. EG: pkg.installed is used to install packages.
  • arguments - Some functions also take arguments. EG: pkg.installed vim

Note

The management IP addresses (10.200.0.0/24) were the only thing configured manually

Note

Both the option and arguments parameters are optional.

Grains

Grain data is collected from the minions by salt automatically and can be used to target hosts and also as variables in device configuration templates.

Use the grains.items function to see all the grains that have been collected from a device.

cmd 

sudo salt '192.168.121.202' grains.items

# output
192.168.121.202:
    ----------
    SSDs:
    biosreleasedate:
        04/01/2014
    biosversion:
        Ubuntu-1.8.2-1ubuntu1
.
. <snip>
.

Use the grains.item <grain-name> function to see a specific grain

cmd 

sudo salt '192.168.121.202' grains.item os_family

# output
192.168.121.202:
    ----------
    os_family:
        Cumulus

Pillar

Pillar data is static variables that relate to minions and is defined by the Salt administrator. The location of the pillar data is defined in the /etc/salt/master config file under the pillar_roots section.

Lets configure some pillar data that defines the minions data centre location to apply to minion configuration. Create two files: master.sls and minion.sls under the /srv/salt/pillar/ directory.

file 

# /srv/salt/pillar/master.sls
data_centre: syd
file 

# /srv/salt/pillar/minion.sls
data_centre: nyc

Note

The management IP addresses (10.200.0.0/24) were the only thing configured manually

Note

By default .sls files are a combination of YAML and Jinja2 data. This means they use the YAML syntax but it's also possible to use Jinja2 template structures to build out dynamic parts of the config file. I will not cover that as part of this blog but its something to be aware of.

The last piece of the pillar puzzle is to configure a top.sls file that ties the pillar data to the minion. Create a file called top.sls under the /srv/salt/pillar/ directory.

file 

# /srv/salt/pillar/top.sls
base: # environment
  '192.168.121.201': # target
    - master # master.sls without the .sls extension
  '192.168.121.202':
    - minion

Use the pillar.items function to see the pillar data avaiable to a minion.

cmd 

sudo salt '*' pillar.items

# output
192.168.121.202:
    ----------
    data_center:
        nyc
192.168.121.201:
    ----------
    data_center:
        syd

Note

The management IP addresses (10.200.0.0/24) were the only thing configured manually

Note

If no data was returned the pillar data may need to be refreshed. This can be done using the saltutil.refresh_pillar function.

cmd 

sudo salt '*' saltutil.refresh_pillar

# output
192.168.121.202:
    True
192.168.121.201:
    True

States

Salt states define the configuration to apply to a minion. Lets create a motd state that configures the message of the day banner using variables from both the pillar and grain data.

The location of the state data is defined in the /etc/salt/master config file under the file_roots section.

Create a file called motd.sls under the /srv/salt/states/ directory.

file 

# /srv/salt/states/motd.sls
/etc/motd:
  file.managed:
    - source: salt://templates/motd.j2
    - template: jinja
    - user: root
    - group: root
    - mode: 0644

Next we need to create a configuration template that will be applied to the minions. The location of configuration templates is defined in the /etc/salt/master config file under the file_roots section.

Salt uses Jinja2 as its default templating engine but it does support many others. Create a file called motd.j2 under the /srv/salt/templates/ directory.

file 

# /srv/salt/templates/motd.j2
############### Salt Managed ###############
hostname: {{ grains['fqdn'] }}
os: {{ grains['os'] }}
os_family: {{ grains['os_family'] }}
{%- if pillar['data_center'] is defined %}
data center: {{ pillar['data_center'] }}
{%- endif %}
############# End Salt Managed #############

Grain and pillar data is available as a dictionary so it can be accessed from within Jinja2 templates using the standard dictionary access methods.

Finally, create a top.sls file under the /srv/salt/states/ directory that defines the states to be applied to minions.

file 

# /srv/salt/states/top.sls
base: # environment
  '*': # target all mininos
    - motd # motd.sls without the .sls extension

Now apply the motd state to the minions with the state.apply command.

cmd 

sudo salt '*' state.apply

# output
192.168.121.201:
----------
          ID: /etc/motd
    Function: file.managed
      Result: True
     Comment: File /etc/motd updated
     Started: 09:40:09.158947
    Duration: 44.262 ms
     Changes:
              ----------
              diff:
                  ---
                  +++
                  @@ -0,0 +1,6 @@
                  +############### Salt Managed ###############
                  +hostname: master
                  +os: CentOS
                  +os_family: RedHat
                  +data center: syd
                  +############# End Salt Managed #############

Summary for 192.168.121.201
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:  44.262 ms
.
. <snip>
.

As you can see below the /etc/motd was updated on both the master and the minion with the variables from the template replaced with the grain and pillar data.

cmd 

# [vagrant@master ~]$ cat /etc/motd
############### Salt Managed ###############
hostname: master
os: CentOS
os_family: RedHat
data center: syd
############# End Salt Managed #############
cmd 

# vagrant@minion:~$ cat /etc/motd
############### Salt Managed ###############
hostname: minion
os: Cumulus
os_family: Cumulus
data center: nyc
############# End Salt Managed #############

Summary

Salt is a very useful tool not just for configuration management but for automation and orchestration. Salt is FAST, secure and once you get you head around it, a pleasure to work with.

Remember: This is not the end, but merely the beginning!

The "from the start to the beginning" series aims to take you from nothing to getting you up and running. This is not meant to be a comprehensive guide, but should be enough to get you going on the journey.

salt