Using Salt with AMP

Objectives

In this tutorial we will show you how to integrate Salt within AMP applications.

Pre-requisites

Some brief explanations of the concepts involved here are provided below, however, it is recommended that you have some experience with Salt in order to use it within AMP. This tutorial assumes you have installed Cloudsoft AMP.

Some Salt Terminology

A ‘State’ in Salt is a specification of a configuration of some aspect of a system, such as what packages are installed, what system services are running or what files exist. Such states are described in SaLt State (SLS) files. These files are typically written as templates using the Jinja templating engine. The actual SLS files for the client computers are then created by processing the templates using configuration data provided via Salt’s “Pillar” system.

Salt comes with built-in support for many software systems, and has a repository of pre-written Salt states known as ‘formulas‘ on GitHub.

How AMP interacts with Salt

AMP provides a Salt entity type which can be specified in a blueprint in order to provision the node through Salt. The Salt entity will download Salt and install it on the node. The entity type supports the configuration of Salt formulas, Pillar data to download, and the configuration of what Salt states to apply. These are managed using Salt in ‘masterless’ mode, as described in the SaltStack quickstart guide, using the ‘salt-call’ command available from Salt. Having installed Salt, the Salt entity will now use it to apply the Salt states specified in the blueprint, thereby installing and configuring whatever the states define.

Once the entity is up and running, the Salt ‘highstate’ (the collection of states applied to the system) is exposed as a sensor on the entity. An effector is provided on the entity that supports ad-hoc Salt operations.

Modes of Use

Salt is an orchestration and configuration management system. Much like Puppet and Chef, Salt is designed to work in a master/agent topology called ‘master/minion’. It also includes a ‘masterless’ mode and an ‘agentless’ topology similar to its competitor Ansible. Salt is (Apache) licensed like Puppet and Chef, whereas Ansible, also a popular choice is GPLv3 licensed.

In the ‘master/minion’ mode, a master server acts as a manager for a number of client nodes, called ‘minions’. A Salt minion daemon connects back to the master server for its operation, and Salt manages the software on the minion according to a specification of ‘states’ defined in Salt configuration files.

In the ‘masterless’ mode, the minion has all the information needed locally, allowing it to run without having to make requests to the master. In this ‘masterless’ mode there is no need for a master, Salt on the minion operates solely based on these local files on the minion node.

The new Salt support in AMP uses this ‘masterless’ mode of operation. AMP manages the remote nodes by running ‘agentless’ Salt commands over an SSH connection.

Instructions

Using Salt in blueprints

To write a blueprint to use Salt with AMP it will help to have a degree of familiarity with Salt itself. In the sections below, when the AMP configuration is described, the underlying Salt operation is also noted briefly, for clarity for readers who already know Salt.

To manage a node with Salt, create a blueprint containing a service of type org.apache.brooklyn.entity.cm.salt.SaltEntity and define the formulas and start_states configuration values.

For example:

name: Salt example setting up Apache httpd
location: my-cloud
services:
- id: httpd-from-salt
  type: org.apache.brooklyn.entity.cm.salt.SaltEntity
  formulas:
  - https://github.com/saltstack-formulas/apache-formula/archive/master.tar.gz
  start_states:
  - apache

This example specifies that AMP should use Salt to download the apache-formula from the Saltstack repository on Github. The Apache formula contains the Apache web server with a simple “it worked” style index page. To start the entity, AMP will use Salt to apply the apache state, which will bring up the web server.

A typical usage of the Salt entity might be to include a formula from the SaltStack repository, such as apache above, and another formula created by the blueprint author, with additional states, such as website content for the Apache server.

A typical Salt deployment will include both state (provided via Salt formulas) and configuration data, provided through Salt’s “Pillar” component. AMP provides configuration keys for the Salt entity to specify where to get the Pillar configuration data. For example:

name: Salt Example setting up MySQL with a Pillar
location: my-cloud
services:
- id: mysql-from-salt-with-my-pillar
  type: org.apache.brooklyn.entity.cm.salt.SaltEntity
  formulas:
  - https://github.com/saltstack-formulas/mysql-formula/archive/master.tar.gz
  - http://myhost:8080/my-mysql-formula.tar.gz
  start_states:
  - mysql
  stop_states: 
  - mysql.disabled
  pillars: 
  - mysql
  pillarUrls:
  - http://myhost:8080/my-mysql-pillar.tar.gz

This blueprint contains the MySQL database, and includes a formula available from myhost which includes the schema information for the DB. The MySQL formula from Saltstack has extensive configurability through Salt Pillars. In the blueprint above, AMP is instructed to apply the pillars defined in the pillars configuration key. (This will add these values to the Salt Pillars top.sls file.) The pillar data must be downloaded; for this, the pillarUrls key provides the address of an archive containing the Pillar data. The contents of the archive will be extracted and put in the /srv/pillar directory on the minion, in order to be available to Salt when applying the pillar. For example, the archive above can simply have the structure:

pillar
├── mysql
│ └── init.sls

The init.sls contains the following pillar configuration values:

Manage databases

database:
  - orders
schema:
  orders:
    load: True
    source: salt://mysql/files/orders.schema

Meanwhile the my-mysql-formula.tar.gz formula archive contains the schema:

my-mysql-formula
├── mysql
│ └── files
│ └── orders.schema

Note that the blueprint above defines an id for the Salt entity. This id, if provided, is set as the minion ID in the Salt configuration file. This is useful particularly for Pillar configuration, as, if there are more than one Salt managed node in the application, they can share a common Pillar file, with appropriate subdivisions of pillar data based on the minion ID.

Highstate Sensors

The Salt entity exposes the Salt “highstate” on the node via AMP sensors. Firstly a single sensor salt.states contains a list of all the top level Salt state ID declarations in the highstate. For example, for the mysql case above, this might look like:

["mysql_additional_config", 
    "mysql_config", 
    "mysql_db_0", 
    "mysql_db_0_load", 
    "mysql_db_0_schema", 
    "mysql_debconf",
    "mysql_debconf_utils", 
    "mysql_python", 
    "mysql_user_frank_localhost", 
    "mysql_user_frank_localhost_0", 
    "mysql_user_nopassuser_localhost", 
    "mysqld"
]

Then, for each ID and each Salt state function in that ID, an AMP sensor is created, containing a map of the data from the highstate. For example, the salt.state.mysqld.service.running sensor might have a value like true.

The Saltcall Effector

The Salt entity includes a general purpose Salt effector, saltCall, which permits execution of Salt commands via salt-call --local. It contains a single parameter, spec, which specifies the command to invoke.

For example, invoking the effector with a spec value of network.interfaces --out=yaml would return a YAML formatted map of the network interfaces on the minion.

Summary

Adding support for Salt integration will ease the process of adopting AMP for projects that already use Salt to manage their deployments, so making it easier to take advantage of AMP’s capabilities.

Explore more Tutorials