Workflow

AMP workflow lets you define operations as part of your Maeztro configuration.

This reference gives some common patterns and tips for use of workflow in Maeztro. Refer to the AMP documentation for Workflow for full information on workflows, including all the available step types and advanced features like concurrency and locks – which can be very useful when working across multiple children Terraform resources in Maeztro!

Writing Workflow in Maeztro/HCL

Workflow is written as a list of steps, where in the simple case each step is a string; this is written in Maeztro (in HCL) as follows:

  steps = [
    "let x = ${try(self.count_sensor, 0}",
    "set-sensor count_sensor = ${x + 1}"
  ]

In more complicated situations, such as setting a map value or invoking a container or using a condition, some steps can be written as maps, with step indicating those attributes which can be set as shorthand:

  steps = [
    {
      step: "let x"
      value: {
        count: "${try(self.count_sensor.count, 0}", 
      }
    },
    "let x.count = ${x.count + 1}",
    "set-sensor count_sensor = ${x}"
  ]

Both examples above increment a sensor’s count; however the first one stores it as a number, whereas the second stores it in a map. This can be very useful if storing multiple keys together in a map.

Note the use of HCL syntax to express steps, lists, and maps.

Using Blocks and List Blocks

To make it easier to write steps, Maeztro adds support for “list blocks” to the HCL language, like normal lists but written as a block and, importantly, allowing blocks within then. This permits constructions such as block [ expression_as_list_item_1, nested_block_as_list_item_2 {} ] and steps [ "let x = 1", "return ${x+1}" ]. It permits : instead of = in blocks, and , as a separator, making it easier to switch between lists and blocks.

Maeztro also interprets the names on a step or steps block, as follows:

• step [ ID ] [ [ "NAME" ] "STEP" ] { ... } - an identifier (unquoted), step (quoted), and optionally name (quoted) between the identifier and step, followed by arguments to the step; the arguments are required but can, and often is, an empty block {}
• steps [ ID ] [ [ "NAME" ] "[" ... "]" - starts a subworkflow with a name and ID, where ... consists of additional step shorthand strings of HCL blocks
• step [ ID ] [ "NAME" ] "STEP" [ ... ] - starts a named step which runs a nested or subworkflow (e.g. foreach, where ... consists of additional step shorthand strings of HCL blocks

Thus step "let x" { value: { count: try(self.count_sensor.count, 0) }} can be used to generate the longhand map step definition, equivalent but more convenient than YAML or HCL map and list (tuple) types.

As a more in-depth example, including an id:

effector "kubectl_apply_all_regions" {
  steps = [
    {
      id : "kubectl_apply_in_regions"
      name : "Run kubectl apply in each region"
      step: "foreach region in ${self.config.demo_regions}"
      concurrency: 3
      steps: [
        {
          name : "Apply at ${region} frontend"
          step : "invoke-effector kubectl_apply"
          entity : "maeztro.${region}_frontend"
        },
        {
          name : "Apply at ${region} app"
          step : "invoke-effector kubectl_apply"
          entity : "maeztro.${region}_app"
        }
      ]
      on-error: [
        "goto kubectl_apply_in_regions"
      ]
    }
  ]
}

can be written as:

effector "kubectl_apply_all_regions" {
  steps [
    step kubectl_apply_in_regions "Run kubectl apply in each region" "foreach region in ${self.config.demo_regions}" {
      concurrency = 3
      steps [
        step "Apply at ${region} frontend"   "invoke-effector kubectl_apply on maeztro.${region}_frontend" {}
        step "Apply at ${region} app"        "invoke-effector kubectl_apply on maeztro.${region}_app" {}
      ]
      on-error [ "goto kubectl_apply_in_regions" ]
    }
  ]
}

List blocks and normal blocks can also be used for on-error steps.

Using Pure YAML

For workflows that aren’t extremely simple, or which are used multiple times, it may be useful to either:

• write the workflow as YAML (as is done elsewhere in AMP, outwith Maeztro)
• store the workflow in a separate file and read it using file

Workflow expressions in steps are normally evaluated when the workflow is run, to ensure current values for sensors are used in above. If you want to force resolution of expressions when the workflow is applied, you can use templatefile and pass the specific variables to use.

An example of this syntax in Maeztro HCL is:

  steps = decodeyaml(file("my-workflow.yaml"))

You can then provide workflow as YAML as is done elsewhere in AMP (and as used in the examples in that documentation):

# file "my-workflow.yaml"
- step: let x
  value:
    count: ${try(self.count_sensor.count, 0)}
- let x.count = ${x.count + 1}
- set-sensor count_sensor = ${x}

Best Practices with Maeztro Workflows

The HCL expression syntax is extremely powerful, and can replace some of the workflow steps and limitations.

For example, without HCL expressions, workflows might be written as:

  "let x = ${self.count} ?? 0",
  "let x = ${x} + 1",
  "set-sensor count = ${x}"

The first step uses the let ?? operator, and the second one uses the arithmetic operators as known to let. With HCL expressions, these can be replaced with the try function and arithmetic inside an expression:

  "let x = ${try(self.count_sensor.count,0) + 1}",    # preferred`
  "set-sensor count = ${x}"

The richer HCL expressions are normally preferred, but two things should be noted.

• Combining HCL expressions with complicated shorthand can confuse the step parser, as step shorthand is not aware of some subtleties such as when ${...} interpolated expressions contain spaces or double quotes. The longhand (map) notation for steps may be required for complicated expressions.

• The workflow idempotency checker assumes that interpolated expressions are idempotent across a step. This may not be the case with HCL expressions: for example, the above steps could be combined as set-sensor count = ${self.count+1}; however if “replayed” from that step, it will increment the sensor twice. By using the let step, the intermediate value is recorded and a replay is guaranteed to be idempotent.

Conditions

Conditions are often used on workflows to indicate whether steps or policies should be run. HCL expressions can be used inside AMP condition/predicate blocks, and in addition, boolean HCL expressions can be supplied as conditions, e.g. condition: ${is_allowed} (instead of the longer workflow condition { target: ${is_allowed}, equals: true }.

These can also be allowed as arguments to if, as in if ${can(x)} then return ${x}, to return ${x} if it exists, using the can function from Terraform.

Executing Commands

• If your organization uses ssh or winrm, these steps can be very powerful in workflows, and Maeztro will automatically configure them according to a Terraform connection block on the resources.
• The container step is recommended for complicated tasks; this uses the kubectl command on the server where Maeztro is running, so all that is necessary to enable this is to configure kubectl.
• The shell step will run on the server where Maeztro is running, if that is supported. This is commonly used in development environments for easier debugging, with a switch to a container if on a server where shell steps are disabled (e.g. if AMP/Maeztro is offered as a service).