One of the first things I needed to get a handle on when I began to use Saltstack was pillar and the various YAML and JINJA formatting techniques. At the time of this writing I didnt see too many beginning to end examples so I thought I would start writing about how I am implementing Saltstack. My intent is to build on this example and end up with a modular deployment using Pillar to manage all the aspects of my infrastructure.
In the following example I will create an ipsec state that we can use to dynamically open ports for our different servers. It allows us to define our open ports in a centralized server configuration file which also allows us to assign different states to our servers (more on this in future articles).
For this setup my entire config will look like this:
/srv/pillar - top.sls - servers.sls /srv/salt - top.sls - linux-server - iptables - init.sls - iptables.jinja
Pillar
In pillar we will create a new servers.sls file which will look similar to the one below. Over time we will use this file to configure various aspects of our servers as well as control what states are applied to each server.
servers: ubuntu00: states: - iptables tcp_ports: - 80 udp_ports: - 5050
Looking at the configuration file from top to bottom we can see that I am first defining the name of the pillar file (servers), then the name of a host (ubuntu00) and under the host I am defining what states will be applied and what ports should be opened for that host.
Next, we will need to add the servers.sls to the /srv/pillar/top.sls so that it can be found by pillar.
base: '*': - servers
State
The state, as its name implies is a new state or configuration that is applied to a minion. To begin my state configuration I am going to create two new directories in my states file server location /srv/salt/linux-server/iptables. I am creating the subdirectory linux-server because I intend to separate my states by host OS and type over time.
mkdir /srv/salt/linux-server mkdir /srv/salt/linux-server/iptables
Next, I need to create a top.sls file at the root of my states tree /srv/salt. I am trying to make this as dynamic as possible, so in the following example I am first declaring a new variable called “hostname” and getting a grains value to populate it, then I am checking for the minions kernel type in order to seperate my windows, mac, solaris and linux configurations into subdirectories named windows-server, linux-server, etc. Lastly I am checking the minions hostname and applying any roles to it that are defined in the servers.sls file we created earlier.
Note: At this time we have only applied the iptables role and have only created a linux-server directory under the salt states root. So for now this example will only work with Linux and the iptables state we are creating.
{% set hostname=grains['id'] %} {% if grains['kernel'] == 'Linux' %} {% set kernel='linux' %} {% elif grains['kernel'] == 'SunOS' %} {% set kernel='solaris' %} {% elif grains['kernel'] == 'Windows' %} {% set kernel='windows' %} {% elif grains['kernel'] == 'Darwin' %} {% set kernel='macintosh' %} {% endif %} base: '*': - common.salt {% if pillar['servers'][hostname] is defined %} {% for roles in pillar['servers'][hostname]['roles'] %} 'servers:{{ hostname }}:roles:{{ roles }}': - match: pillar - {{ kernel }}-server.{{ roles }} {% endfor %} {% endif %}
Now we need to setup the state by creating a file named init.sls in the /srv/salt/linux-server/iptables directory.
If we examine the init.sls from top to bottom we can see it is first declaring a variable named “path” to define different paths to the iptables file on Debian and RedHat systems. Please note: This is Jinja2 markup and all command blocks are surrounded in {% %} while all variable blocks are declared with {{ }}.
Next we find the YAML markup where we are defining our state. Here we are:
- Naming the state: iptables
- Calling the pkg command and making sure the package iptables is installed.
- Calling the file command and using the shortcut .managed to declare that a file should be managed.
- Declaring the name of our file and using the variable set at the top of the init.sls file
- Set owner of file
- Set group of file
- Set permissions on file
- Define where the file should be transferred from using the salt file server (We will create this file in the next step)
- Declare that the file’s template should use jinja2
- Calling the cmd state command with the .wait shortcut to declare that we should run a command only if the iptables file as been changed.
- Define the command we want to run, which in this case is “iptables-restore <” against the iptables file we transferred over.
- Here we are watching for changes to the iptables file before we run the command.
- Here we are defining that this requires us to have the iptables package installed on the minion.
{% if grains['os_family'] == 'Debian' %} {% set path='/etc/network/if-pre-up.d/iptables' %} {% elif grains['os_family'] == 'RedHat' %} {% set path='/etc/sysconfig/iptables' %} {% endif %} iptables: pkg: - installed file.managed: - name: {{ path }} - user: root - group: root - mode: 755 - source: salt://linux-server/iptables/iptables.jinja - template: jinja cmd.wait: - name: iptables-restore < {{ path }} - watch: - file: iptables - require: - pkg: iptables
Lastly we are going to define our iptables template. For the purposes of this example I took a default iptables dump from a linux box and added the JINJA formatting to first set the variable “hostname” using grains and then towards the bottom I start looping through the pillar data in my servers.sls in order to open the appropriate ports.
{% set hostname=grains['id'] -%} # This file is managed by Saltstack. Please configure from the Master server. *filter # Drop any traffic not explicitly allowed in the rules below. :INPUT DROP :FORWARD DROP :OUTPUT ACCEPT # Allow all loopback (lo0) traffic -A INPUT -i lo -j ACCEPT # Drop all traffic to 127/8 that doesn't use lo0 -A INPUT ! -i lo -d 127.0.0.0/8 -j DROP # Accept inbound traffic for already established connections. -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT # Effectively allow all outbound traffic. -A OUTPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT # Allow ping -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT # Allow connection to the services running on this server. -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT {%- if pillar['servers'][hostname] is defined %} {%- if pillar['servers'][hostname]['tcp_ports'] is defined %} {%- for port in pillar['servers'][hostname]['tcp_ports'] %} -A INPUT -p tcp --dport {{ port }} -j ACCEPT {%- endfor %} {%- endif %} {%- endif %} {%- if pillar['servers'][hostname] is defined -%} {%- if pillar['servers'][hostname]['udp_ports'] is defined -%} {%- for port in pillar['servers'][hostname]['udp_ports'] %} -A INPUT -p udp --dport {{ port }} -j ACCEPT {%- endfor %} {%- endif %} {%- endif %} COMMIT ## If the firewall needs to be disabled, run the following command: ## ## iptables-save | sed "/-/d;/^#/d;s/DROP/ACCEPT/" | iptables-restore
Very interesting post! I’m quite new to Salt and was running through this but got stuck with the following error:
SaltRenderError: Undefined jinja variable; line 15 in template
Which corresponded to:
{% for roles in pillar[‘servers’][hostname][‘roles’] %}
What I cannot figure out is where the “roles” value is being pulled… at first I thought it was pulled from the states section of the servers.sls file but if so it’s not referencing right… then I thought perhaps it should be states not roles but that doesn’t work…
I also had to remove the – common.salt line as that doesn’t correspond to anything as far as I can tell.
I’m sure I’m doing something stupid/ignorant etc but I’ve been straining my brain and I just can’t see it. Any thoughts?
Thanks again,
David
Hi David,
It looks like I should have put in the servers.sls pillar in the documentation. You can find the full state / pillar on my github. I will make sure to fix that as soon as I have a second. Sorry for the headache!
https://github.com/gl1ch/salt-relational/tree/master/pillar
As for the common.salt, this is another section that I use to apply common states to multiple machines which you should also be able to find on my github.
Thanks
Andrew
Thanks for the great tutorial. You really helped me get my head around pillars (especially after looking round your github).
I’ve a small question to ask though. In the above example how would I add options to the iptables statement?
For example, how would I include the following:
-A INPUT -p tcp –dport 80 -m limit –limit 25/minute –limit-burst 100 -j ACCEPT
Thank you for any help
Or for that matter, allowing connection to a port only from a specific IP. Sorry if this is simple and I am asking stupid questions.
This guide helped me a lot understanding salt, pillars and jinja. And iptables a bit. I just wanted to thank you for posting this!