Sensors, Widgets, and Policies

Sensors

AMP “sensors” store runtime information on resources. They can come from any source, retrieved on demand or periodically, or can be pushed by external systems. They can hold metrics from monitoring source, additional runtime state from the component or a cloud provider, or any relevant data from other systems, including CMDBs, issue trackers, and source control.

Steps, Output, Triggers, and Period

A sensor is defined in a sensor "NAME" block and should define either an output value or steps to run.

The simplest sensor will evaluate an expression and return it as output:

sensor "main_uri" {
  output = "http://${self.public_ip}:${var.port}/"
}

Most sensors will be more interesting and use workflow steps, for example to call a REST endpoint:

sensor "error_count" {
  steps = [
    "http http://${self.main_uri}/metrics?error_count",
    "return ${content_json}"
  ]
}

The output of any return step, or of the last command, is the value set on the sensor: the content_json field is used above, and it is defined as one of the outputs from the http step. If an output attribute is specified alongside steps, its value is computed after the workflow steps run, with all workflow variables and the output of the last step in scope. Thus example above could also be written as:

sensor "error_count" {
  steps = [ "http http://${self.main_uri}/metrics?error_count" ]
  output = content_json
}

Alongside steps or output (or both), sensor blocks typically also specify either a period of how often it should run, or a list of sensors that act as triggers, or both:

sensor "error_count" {
  steps = [
    "http http://${self.main_uri}/metrics?error_count",
    "return ${content_json}"
  ]
  triggers = [
    "main_uri"
  ]
  period = "1m"
}

The example above will run every minute and whenever the main_uri sensor changes. The syntax for period is any number followed by any valid unit, either ms, s, m, h, or d, and any space separate combination using different units, e.g. 1h 30m. The syntax for triggers is a list where each element is either a string, to indicate a sensor on the local entity, or a map containing an entity reference to a resource address and the name of a sensor on the entity.

A type can be set on a sensor block to enforce that the value be coercible to that that type, and retrieved as that type. For example setting integer on the above will ensure any attempt to read or write will use an integer or throw an error.

It can also be useful to define a sensor without steps or output, if that sensor is expected to be set by another source – either pushed to AMP from an external source via AMP’s REST API, or set by a policy. This is not necessary, but it makes the sensor discoverable on the resource even if it is not set, and it can be used to enforce a type.

As described on effectors, there are several other attributes supported on sensors, including a condition to prevent running and setting a sensor when the condition is not met; along with on-error, lock, and timeout. The parameter block is not applicable to sensors, as sensors run automatically.

Full details on sensors are available in the [../reference/adjuncts.md].

Widgets

The widget block is used alongside sensors to display a sensor value or Terraform attribute.

In its simplest form, it can simply refer to a sensor as the block ID:

widget "main_uri" {}

The example above will display the value of the main_uri sensor defined above, with a label main_uri.

The widget block also supports setting an explicit label, value, and unit. The value can be any expression evaluated at the resource entity, e.g. value = Note that even if label and value` are supplied, the block ID must be supplied so that the source of the definition is uniquely identifiable by Maeztro, in case it is updated or removed.

Full details on widgets are available in the [../reference/widgets.md].

Policies

AMP “policies” describe the control and governance logic to run for a system or resource under management. Structurally these are similar to sensor blocks: they must supply an ID, define steps, and define at least one of period or triggers.

In usage, however, policies tend to be more significant than sensors, and typically they effect some change in an external system or in the AMP model, either by invoking effectors through workflow or running sophisticated workflow steps.

Policies also accept condition, on-error, lock, and timeout attributes.

The following example will invoke a container slack_notifier whenever the error count exceeds 10, passing CHANNEL as an environment variable and a message as an argument:

policy "slack_on_high_error_count" {
  condition = {
    target = self.error_count
    greater-than = 10
  }
  steps = [
    {
      step : "container slack_notifier"
      env : {
        CHANNEL : "sre_ops"
      }
      args: [
        "Maeztro detected ${self.name} has high error count (${self.error_count})"
      ]
    }
  ]
}

Full details on policies are available in the [../reference/adjuncts.md].