cloudsoft.io

Common Types

Compute Node

The tosca.nodes.Compute node type defines a compute resource, e.g. a VM provisioned in VMware or a cloud, or a pre-existing “BYON” (bring-your-own-node) machine. If necessary, it is created during the create step and can run scripts as part of the configure or start operation, as well as stop. By default it shuts down but keeps a provisioned machine on stop (or suspends it, for cloud machines, where shutdown is not universally available), so that it can be restarted/resumed on a subsequent start, and it destroys it on delete.

The following common capabilities are relevant, with limitations and/or extensions as noted:

  • host (tosca.capabilities.Compute) - as per TOSCA spec, supporting mem_size, disk_size, and num_cpus in most environments, and cpu_frequency and name in some; this is not restricted to the normative valid_source_types: [tosca.nodes.SoftwareComponent]

  • os (tosca.capabilities.OperatingSystem) - not supported; the TOSCA spec is ambiguous about how to specify base images (auto-detect vs specify an image), the filtering it requires is typically not supported by cloud platforms; and the filtering is not typically powerful enough for common use cases (e.g. take the “most recent LOB-approved image with Ubuntu”); in line with the spec, this implementation expects images/OVA/ISO/Template specified as artifacts which should be passed to the create operation, as described below

  • endpoint (tosca.capabilities.Endpoint.Admin) - not supported

  • scalable (tosca.capabilities.Scalable) - not supported, due to ambiguity; Scalability can be supported however by grouping templates using a tosca.entity.DynamicCluster and attaching AMP Policies to node templates.

  • binding (tosca.capabilities.network.Bindable) - not supported due to the ambiguity in the spec around re-declaring networks; many common features are available via the cloudsoft and vmware extensions

Cloudsoft Generic Extension Capabilities

  • cloudsoft - the following extensions exposed by the underlying Cloudsoft AMP system are commonly used in many clouds (but not for use with VMware); see Cloudsoft / Jclouds Capabilities for more information:
    • required.ports (java.lang.Object): Required additional ports to be applied when creating a VM, on supported clouds (either a single port as an Integer, or an Iterable or Integer[])
    • securityGroups (java.lang.Object): Security groups to be applied when creating a VM, on supported clouds (either a single group identifier as a String, or an Iterable or String[])
    • imageId (java.lang.String): A system-specific identifier for the VM image to be used when creating a VM
    • imageNameRegex (java.lang.String): A regular expression to be compared against the ‘name’ when selecting the VM image to be used when creating a VM
    • extraSshPublicKeyData (java.lang.String): Additional public key data to add to authorized_keys (multi-line string supported, with one key per line)
    • installDevUrandom (java.lang.Boolean): Map /dev/random to /dev/urandom to prevent halting on insufficient entropy
    • disableRootAndPasswordSsh (java.lang.Boolean): Whether to disable direct SSH access for root and disable password-based SSH, if creating a user with a key-based login; defaults to true (set false to leave root users alone)
    • dontRequireTtyForSudo (java.lang.Boolean): Whether to explicitly set /etc/sudoers, so don’t need tty (will leave unchanged if ‘false’); some machines require a tty for sudo; AMP by default does not use a tty (so that it can get separate error+stdout streams); you can enable a tty as an option to every ssh command, or you can do it once and modify the machine so that a tty is not subsequently required. Usually used in conjunction with ‘dontCreateUser’ since it will prevent jclouds from overwriting /etc/sudoers and overriding the system default. When not explicitly set will be applied if ‘dontCreateUser’ is set.

VMware Capabilities

When provisioning via VSphere, the vmware capability can be used to provide extensive configuration. The following properties are supported on this capability:

  • cluster (java.lang.String): The name of the ClusterComputeResource to use; if not supplied, it will auto-select
  • datastore (java.lang.String): The name of the datastore to use; if not supplied, it will auto-select
  • resourcePool (java.lang.String): The name of the resourcePool; if not supplied, it will auto-select
  • customIpAddress (java.lang.String): IP address to request/assign for this instance; typically leave blank and have vCenter choose
  • customHostname (java.lang.String): If you want to set a custom hostname otherwise leave blank and use default depending on OS, Required ifcustomHostName set.
  • customDomain (java.lang.String): If you wish to set a custom domain otherwise leave blank and use default depending on OS. Required on linux if CUSTOM_IP set
  • customSubnetMask (java.lang.String): If you wish to set a custom subnet mask otherwise leave blank and use default depending on OS. Required ifcustomHostName set
  • customNameservers (java.lang.String[]): If you wish to set a custom DNS list otherwise leave blank and use default. Required if CUSTOM_IP set
  • osFamilyOverride (org.jclouds.compute.domain.OsFamily): OS family of VMs (ignores VM metadata from VMware, and assumes this value); common values include: “windows ““linux”, “debian”, “centos”, “ubuntu”, “rhel”
  • windowsProductId (java.lang.String): If you want to set a custom ip or hostname for a windows based machine you need to also configure the product id otherwise a default of XXXs isset which will eventually fail if not then corrected later
  • customWindowsWorkgroup (java.lang.String): Set the workgroup that this server should join - defaults to WORKGROUP
  • imageId (java.lang.String): A system-specific identifier for the VM template image to be used when creating a VM
  • imageNameRegex (java.lang.String): A regular expression to be compared against the ‘name’ when selecting the VM template image to be used when creating a VM
  • windowsTimezone (java.lang.Integer): Microsoft timezone - see https://docs.microsoft.com/en-us/previous-versions/windows/embedded/ms912391(v=winembedded.11) - defaults to GMT
  • windowsUserOrganisation (java.lang.String): Full name of the end user. Note that this is not the username but full name specified in “Firstname Lastname” format”
  • windowsUserFullName (java.lanf.String): Name of the organization that owns the computer.
  • customGateways (java.lang.String[]): List of gateways to set on adapter
  • additionalVolumes (List<StorageVolumeDefinition>): List of maps containing keys size (e.g. "100 gb") and optional order (if supplied each of the n values must be unique in the range [0,n-1])
  • managementNetworkName - the network via which AMP can access the VM under management. If this is not set then the order 0 NIC will be used.
  • publicNetworkName - the public network. If this is not set then the management network will be used.
  • networks (List<NetworkDefinition>): List of maps describing networks, as described below

The create operation on the Standard interface can be customized with an Image artifact to specify a given image to use when creating the host.

The normative attribute sensors public_address and private_address are set on the node entity instance,

On Windows machines, two additional sensors are available – windows.username and windows.password.encoded – because many Windows scripts require this information (e.g. in order re-authenticate); the latter one is base-64 encoded to prevent odd characters causing problems and casually exposing it.

VMware Networks

The NetworkDefinition object used in the networks property accepts the following fields:

  • mandatory name (the name of the network in VMware)
  • optional order (as per additionalVolumes)
  • optional dhcp (described below)
  • optional ipAddress
  • optional subnetMask
  • optional gateways (list of ip addresses, e.g. ["10.0.0.1", "10.0.0.2"])
  • optional nameservers (list of ip or ipv6 addresses, e.g. ["10.0.0.1", "10.0.0.2"]); on Linux these are aggregated and set globally because per-NIC DNS is not available
  • optional ipV6Auto (described below)
  • optional ipV6Address
  • optional ipV6SubnetMask (unlike ipv4, this is the number of initial 1 bits, so 64 not ffff:ffff:ffff:ffff::)
  • optional ipV6Gateways (list of ipv6 addresses)

If the networks property is used, then the simpler customIpAddress and customSubnetMask must not be set.

Either or both IPv4 or IPv6 are supported on each network. The default is to use DHCP for IPv4 unless IPv6 is configured, where IPv6 is configured only if an explicit ipV6Address is provided or ipV6Auto is set to auto.

The options for dhcp are as follows:

  • auto: DHCP will be used to provision an IPv4 address only if nothing is specified for IPv4 or IPv6 (the default)
  • enabled: DHCP will be used to provision an IPv4 address if nothing is specified for an IPv4 address
  • required: provision an IPv4 address with DHCP - will fail if ipAddress or other fields are set
  • disabled: DHCP is disallowed - will fail unless ipV6 is enabled or an ipAddress and other IPv4 fields are not set

The options for ipV6Autoare as follows:

  • no_auto: IPv6 auto-configuration (RADVD/DHCP) will not be used; IPv6 will only be configured if an explicit ipV6Address is supplied (the default)
  • auto: IPv6 is enabled and will be autoconfigured (RADVD/DHCP) if nothing is specified for an ipV6Address
  • REQUIRED: IPv6 is enabled and must be provisioned automatically - will fail if ipV6Address or other fields are set
  • manual: IPv6 is enabled but must be provisioned explicitly - will fail if ipV6Address is not set
  • no_ipv6: IPv6 will not be configured - will fail if ipV6Address or other fields are set

In addition to the normative attribute sensors public_address and private_address, when multiple networks are configured, additional attribute sensors are published with more information about multiple addresses. For example if 2 NICs are configured then 2 extra sensors will be created as follows.

  • network.network-1.ip_address
  • network.network-2.ip_address

where network-1 and network-2 will be replaced with the names of the attached networks.

VMware Timeouts

It is recommended that VMware templates boot quickly and do not do excessive setup on first boot. Timeouts are in place to detect and prevent errors, as shown below. These can be overridden as vmware capability properties if required:

  • powerState.on.timeout: Max time to wait for power to be confirmed as on, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 2m
  • powerState.off.guestShutdown.timeout: Max time to wait for power to be confirmed as off following a guest shutdown call, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 10m
  • powerState.off.hardShutdown.timeout: Max time to wait for power to be confirmed as off following a hard power-off call, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 2m
  • powerState.off.additionalDelay: Delay between a power state off confirmation from VMware and proceeding assuming it is off (eg to deletion), because there seems to be some eventual consistency at play, default 3s
  • creation.additionalDelay: Optional delay after VM created before using or performing any checks (default none; useful if other VMware checks such as CustomizationSucceeded and expected IPs report readiness too early and/or are skipped), default 0
  • guestTools.timeout: Max time to wait for VMware guest tools to be reported as operational, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 15m
  • customizationStart.timeout: Max time to wait for VMware customization to start, skipping waiting for completion if not started unless indicated as required, default 5m
  • customizationStart.required: Whether to fail if customization does not start within the customizationStart.timeout period, default false
  • customizationComplete.timeout: Max time to wait for VMware customization to be reported as complete, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 1h
  • expectedIps.timeout: Max time to wait for expected IPs to be reported from VMware, failing if not confirmed unless ‘false’ provided here (and won’t wait), default 15m

Software Component Node

The tosca.nodes.SoftwareComponent node type is straightforward to use. Typically, as shown in the examples, it has a host relationship to a tosca.nodes.Compute node, and defines operations according to the tosca.interfaces.node.lifecycle.Standard lifecycle interface (create, configure, start; stop; delete) and tosca.interfaces.relationship.Configure interfaces on relationships ({pre,post}configure{source,target}`, as described below.

Interfaces can define and extend operations using the operation and artifact types as defined in the following section.

Host Requirement

As per the TOSCA spec, a host requirement can be supplied pointing at a Compute node.

As an extension for convenience, if a software component is defined without a host specified, a host will automatically be created according to the default tosca.default.location setting. This can greatly reduce the size of blueprints in many cases, without impacting users who wish to define hosts or define custom requirements.

As a further convenience, the capabilities used for provisioning a Compute node can also be specified on a Software Component node. This applies whether the host requirement is explicit or the implicit convenience is used. In the case of RAM, disk size, and ports, the largest value is used. In other cases it is an error if incompatible (different) values are used.

Interfaces and Operations

The tosca.interfaces.node.lifecycle.Standard lifecycle interface is the primary way that node behavior is configured: create, configure, start; stop; delete. In addition the tosca.interfaces.relationship.Configure interfaces on relationships ({pre,post}configure{source,target}` allows nodes with relationship to get additional configuration steps. (The other relationship operations, triggered on changes, are not supported because of ambiguity in making changes.)

These are supported on most nodes, including SoftwareComponent and Compute (with the create step of Compute extended to actually create a VM).

A capability of type brooklyn.tosca.capabilities.StandardLifecycleBehavior can be declared to configure the behavior of start / stop / delete and up-ness and heath checks, as follows:

  • quorum_up: how many of a node’s children must be up for a node to report itself as up; valid values are all, atLeastOne, allAndAtLeastOne, and alwaysHealthy (no children required), or a list of numeric pairs [ [0,0],[2,2],[10,6] ] where each pair’s first co-ordinate is the number of children defined, increasing, and the second pair indicates for that number of children how many must be up, with linear interpolation (so the example requires 100% of 2 nodes, 60% of 10 nodes, explicitly, and by interpolation 67% of 6 nodes and 60% beyond 10); default is alwaysHealthy for most nodes, so that where children are hosted on a parent the parent doesn’t also wait for children (!); however nodes which do nothing except aggregate their children can safely and usefully default to all or another quorum expression; for example Collection (below) defaults to all, and topology_template applications also use all behavior
  • quorum_healthy: how many of a node’s children must be healthy for a node to report healthy; this differs from quorum_up in that only children which are running or themselves unhealthy are considered; default is all for all nodes (options are the same as quorum_up)
  • stop_waits_on_dependents_timeout: whether stop should wait for all dependencies to stop, taking either a boolean or a timeout, e.g. 1h to have it wait up to one hour for dependencies to stop before deleting (default true for most node types, as e.g. shutting down a Compute node while running scripts to stop software is clearly not going to work well); note that stop in not typically invoked on dependencies, but instead it is assumed the operator will be stopping in sequence or a parent topology will stop all nodes in parallel and this capability allows sequential resolution; also note if A depends on B and B depends on A, this behavior can cause them to block on each other, so at least one should have this capability set false if the co-dependency is correct
  • delete_on stop: whether stop should invoke delete (default false)
  • delete_invokes_stop: whether delete should stop the node first (default false)
  • delete_waits_on_dependents_timeout: whether and for how long delete should wait for all dependencies to be deleted, either a boolean or a duration (default true for most node types meaning to wait indefinitely, but overridden for Compute to be false as if you are deleting an instance there is no need normally also to delete the software on the box; can of course be overridden for specific instances or subclasses of Compute)
  • restart_pause: when doing a restart, how long to wait between stop completing and start being invoked; typically this should be long enough for dependent nodes to change state to prevent race conditions when restarting multiple nodes with dependencies simultaneously; default 5s

(The deprecated capability brooklyn.tosca.capabilities.StopDeleteBehavior is accepted as a synonym for this.)

The operations to start, stop, and delete are recursive; if children nodes have the same effector, they will normally be invoked as part of the parent’s {start,stop,delete}. Where a child does not have such an effector, behaviour will typically not recurse into children: if combining nodes which support delete with external (non-TOSCA) nodes that do not have a delete effector, it might be necessary to have delete_invokes_stop true. For topology templates, the default behavior of delete is to invoke delete children which have a delete effector and then to invoke stop on children which have a stop effector, to facilitate compatibility with nodes that do not offer delete.

Node Management Hierarchy

As an extension to TOSCA, building on Cloudsoft AMP management functionality, nodes can be logically grouped underneath other nodes as “parents”. This does not affect the basic start/stop functionality, but it does give several benefits:

  • In the design-time blueprints in Composer and runtime models in Inspector, nodes are shown underneath their parents; this makes blueprints with many nodes easier to work with, because the groups separate different logical elements (for example different tiers)
  • Invoking start / stop on a logical parent will invoke that operation on its children, in parallel with siblings and the parent (with blocking possible where there is a dependency); this allows operations after the initial deployment to be easily carried on selected nodes as a unit
  • The quorum checks (see previous section) allow a parent node’s state to be computed based on its state and the state of its children

This logical management parenting/grouping is done using the group type brooklyn.groups.Parent with the TOSCA group members being the intended children, and a parent property indicating the ID of the parent node.

While any node can be a parent, the special node type brooklyn.tosca.nodes.Collection is available to facilitate grouping without assigning any behavior to the parent other than applying operations to its children. This is often useful for aggregated reporting and operations without the ambiguity which can occur when the parent has additional behavior. For example a “host-parent” modelling approach, where a compute node is a parent of software components hosted on it, is familiar where compute nodes are long-lasting, but it has some limitations: there is no easy way to start the compute node without starting the software component, and it is difficult to distinguish a software-only error because the parent compute node will reflect the error. An alternative “host-child” approach, where a compute node is the child of a software component hosted on it, is not uncommon where the compute node is a comparatively unimportant facet of the software, but again there are limitations: there is no easy way to stop the software process without stopping the compute node. Either of the above are acceptable, so long as the limitatons are understood; however if all elements are wanted to be explicitly modeled, an approach where hosts and dependents are siblings underneath a Collection is the way to go.

The template below shows the use of the Parent group and a Collection node containing sibling compute and software components:

topology_template:
  node_templates:
    parent-collection:
      type: brooklyn.tosca.nodes.Collection
    child-node1:
      type: tosca.nodes.Compute
    child-node2:
      type: tosca.nodes.SoftwareComponent
      requirements:
      - host: child-node1 
groups:
- parent-collection-children:
    type: brooklyn.groups.Parent
    members: [child-node1, child-node2]
    properties:
      parent: parent-collection

Operations and Artifact Types

Cloudsoft AMP supports the following artifact types within operations:

  • tosca.artifacts.Implementation.Bash: files of type *.sh are invoked on remote machines via SSH. If a shell script explicitly calls exit 0, the output environment variables will be unavailable. If a shell script calls exit Xm where X is not 0(ZERO) the child and the parent task fail, the deployment stops and outputs are not available.

  • tosca.artifacts.Implementation.Python: files of type *.py are invoked on remove machines via SSH, installed as main.py and run using python main.py; the OS image must have the desired version of python already installed and on the path and should support bash scripts

  • brooklyn.tosca.artifacts.Implementation.WinRm.{PowerShell,Cmd,Bat}: files of type *.ps1, *.cmd, and *.bat are invoked on Windows machines using WinRM; formats are preferred in that order; outputs may not be available for non-preferred formats; If a script explicitly calls Exit 0 the exitCode and LASTEXITCODE output environment variables are available. If a shell script calls Exit Xm where X is not 0(ZERO) the child and the parent task fail, the deployment stops and outputs are not available.

  • brooklyn.tosca.artifacts.Implementation.WinRm.Reboot: this artifact type indicates that a windows machine should be rebooted; artifact files, dependencies, and inputs are ignored; properties force (a boolean) and sleep (a duration, default 0 min which just polls the VM) are accepted

  • brooklyn.tosca.artifacts.Implementation.CredentialsUpdate: this artifact type indicates that the provided crededentials should be used for subsequent operations; artifact files, dependencies, and properties are ignored; inputs login (string), and password (string) are expected. For example, when using CredentialsUpdate as part of an InitD statement (below), you could use the following:

    01-UpdateCredentials:
      implementation:
        primary: 
          type: brooklyn.tosca.artifacts.Implementation.CredentialsUpdate
      inputs:
        login: { get_property: [ SELF, login_1 ]}
        password: { get_property: [ SELF, password_1 ]}
    
  • brooklyn.tosca.artifacts.Implementation.Initd: this artifact type bundles multiple operations together in a single operation, in a property commands which takes (usually as its default value) a map of step IDs to an operation definition in either longhand (an operation definition map, including inputs and an implementation) or shorthand (an artifact file or type, as a string); the step IDs are executed in alphanumeric order (the reason for this is that it allows subclasses to interleave or overwrite steps). Note that artifact files and dependencies on this artifact are ignored, but where defined on a nested operation they are respected, and outputs are taken from the last operation only.

  • brooklyn.tosca.artifacts.Implementation.NoOp: indicates that the step should do nothing; although the TOSCA spec allows operations with no implementation, that can frequently be a sign of an error as use cases for no-op operations are rare; it is advised that operations should always give an implementation, and those operations which are intended to be no-op should declare this implementation artifact; in this implementation of TOSCA an error is thrown if an operation is provided without an implementation; the main use cases for this artifact are where a parent behaviour wants to be suppressed, or where an artifact is needed in conjunction with inputs which reference a get_attribute to indicate that a step should block on an attribute being available

  • tosca.artifacts.Implementation.AnsiblePlaybook: ansible playbooks of type *.yaml are invoked via SSH on a pre-existing BYON ansible machine targeting the node that declared it. The ansible machine is expected to be configured in tosca.ansible.default.location setting which requires the public SSH key data of the ansible machine set in publicKeyData property, along with hosts and user. Ansible machine is expected to have ansible-playbook installed with default access permissions as well as sort and uniq. The artifact declaration accepts boolean become property which is used as --become argument for ansible-playbook if set to true, extra_vars property of string type as --extra-vars argument and additional_arguments as a list of strings property for other arguments. Note that additional_arguments does not accept --extra-vars, --become or --inventory arguments. --inventory argument is reserved to target the node that declared this artifact, it means that inventory configured on ansible machine will be ignored.

    artifacts:
      playbook_artifact_example:
        file: classpath://playbooks/playbook.yaml
        type: tosca.artifacts.Implementation.AnsiblePlaybook
        properties:
          extra_vars: ansible_python_interpreter=/usr/bin/python3
          become: true
          additional_arguments: ["-vvv"]
    

Repositories support

When artifacts are hosted on a secure repository (e.g. Artifactory), the repository should be declared using the repositories TOSCA configuration. Private repositories require credentials which are provided by the credential configuration element of type tosca.datatype.Credential. The credential configuration is not mandatory, and when its value is not set the repository is assumed to be public. Repositories have a unique ID and are stored in the catalog. Thus, can be used in all topology_template deployed with AMP by referencing their id. The following code snippet shows two repositories being declared and then used in a topology_template.

repositories:
  cloudsoft_artifactory_repo:  # private repo ID
    description: Cloudsoft Artifactory Repo
    url: https://artifactory.cloudsoftcorp.com/artifactory
    credential:
      user: xxx
      token: xxx
  public_maven_repo:  # public repo ID
    description: Public Nexus Maven Repo
    url: https://repo1.maven.org/
    # no credentials

# usage
topology_template:
  node_templates:
    example1:
      type: example.CustomCompute
      artifacts:
        simple_script:
          file: libs-release-local/io/cloudsoft/scripts/create.sh
          repository: cloudsoft_artifactory_repo  # private repo ID
          type: tosca.artifacts.Implementation.Bash
          deploy_path: /tmp/create.sh
        simple_jar:
          file: /maven2/junit/junit/4.13.2/junit-4.13.2.jar
          repository: public_maven_repo  # public repo ID

Inputs, Outputs, and Artifacts

Unless otherwise indicated, inputs are set for all artifacts, dependencies are copied across either to the indicated deploy path or else inferred from the dependency file (URL). Where dependencies are identified by a type name or ID in the deployment list, an environment variable is set with the name pointing at the deployed path. Less common and more ambiguous options, such as operation_host and timeout, are not typically supported unless otherwise indicated for an operation.

Files in artifacts can be either URLs or absolute paths to files within the CSAR. Where node types are extended, paths within the CSAR are resolved relative first in the extension’s CSAR bundle and then to CSAR bundles where ancestors are defined. Relative paths to files within the CSAR are not supported due to ambiguity (if a node type is extended, especially in another CSAR) and because they can be brittle.

Inputs can take the TOSCA DSL (e.g. get_input, get_property, get_attribute, etc); however shorthand and assignment syntax cannot be used with the DSL. If using the DSL, the value must be set under the keyword default within the input.

Outputs are taken from the new environment variables that are set after execution; for instance if a bash script does an export DATE=$(date), an output called DATE will be available on the operation. These are sent as “sensors” namespaced by the operation within the runtime. The name of the interface definition (not the interface type, although often this matches the type’s short name, e.g. Standard) must be used when using get_operation_output. Behaviour of artifacts on relationship operations is fully supported with the exception that get_operation_output is not supported for the output from relationships, for multiple reasons (relationship templates can have multiple instances, so a reference to a relationship instance is not necessarily well-defined; it would be well-defined for SELF except the spec says SELF is only for node templates; and in the runtime model relationships are simpler than nodes and do not have sensors).

It is recommended that outputs be mapped to attributes on the entity, using the syntax outputs: [ SELF, <attribute_name> ], and get_attribute: [ SELF, <attribute_name> ] be used to access them, rather than use get_operation_output. This is because the latter buries information which is usually important and which by definition is needed across operations, and introduces brittleness sensitive to the interface and operation identifiers.

This implementation of TOSCA adds support for an output_defaults map where an output name and default value can be set; these are used if the artifact does not itself produce outputs.

For usage examples and additional notes, view the Operation Examples.

On-Box Script Execution and Deletion

Scripts are executed underneath ~/.brooklyn-tosca-execution, should for debugging purposes it be desired to access the box and inspect behaviour. The actual structure within this is <application-id>/<entity-id>/date>-<op>/.

By default most script files, outputs, and artifacts are kept for the lifetime of a node, and deleted automatically if and only if there is a delete command on the node.

There is an implicit property brooklyn.tosca.operations.keep_files which can be set true to force script files to be kept, or false to force them to be deleted. This can be set on an operation or an interface (as an input), or on a node (as a property), with more tightly scoped settings being preferred to more generally scoped settings. By default, most files are kept while the node is active, with some exceptions (such as check-running, which runs very frequently), and then deleted by the platform if and when a delete command is run. It is normally not necessary to change this unless commands are run frequently or the target machine is kept for a long time. A cron job can also be set to delete old scripts if it is desired to keep them for some other reason. The application and entity IDs are available as attributes on the node.

Checksum Validation

If a checksum value is configured for a artifact, validation is performed on target machines when installed as part of bash, powershell, and related operation implementations. As per the TOSCA spec, the checksum algorithm should also be explicitly configured on any artifact supplying a checksum, using the checksum_algorithm property.

To perform the validation, this TOSCA implementation expects a specific command on the target machines. The command takes 3 arguments, <file> <checksum> <checksum_algorithm>. The <file> is placed on the target machines by the operations and is guaranteed to exist, and the <checksum> and <checksum_algorithm> arguments are exactly as supplied in the TOSCA (case sensitive) and guaranteed by the framework to be non-blank. The scripts themselves are typically baked in to images used with this TOSCA server. Sample such scripts for Linux and Windows are provided below.

The script name can be configured by specifying the keys brooklyn.tosca_verify_checksum.cmd.linux and brooklyn.tosca_verify_checksum.cmd.windows, e.g. in the /etc/brooklyn.cfg or brooklyn.properties. By default, on Linux it expects the script tosca_verify_checksum on the path, and on Windows it expects the script in tosca_verify_checksum.ps1 located under C:\Scripts.

To simplify validation for the most common algorithms, this TOSCA implementation supports some common algorithms on Linux without the above script present, using default on-box tools: MD5, SHA1, SHA256, and SHA512 (using md5sum, sha1sum, sha256sum, and sha512sum respectively). For other checksum algorithms the script is required, and in other cases it is often recommended, as it
gives organizations complete control of the checksum algorithms supported depending on their requirements.

Example tosca_verify_checksum script for Linux:

#!/bin/bash

FILE="$1"
CHECKSUM="$2"
ALG="$3"
ALG_UPPER=$(echo "${ALG}" | tr '[:lower:]' '[:upper:]')

case ${ALG_UPPER} in
   MD5)
    COMPUTED_CHECKSUM=$(md5sum "${FILE}" | cut -d\  -f1)
    ;;
  SHA1)
    COMPUTED_CHECKSUM=$(sha1sum "${FILE}" | cut -d\  -f1)
    ;;
  SHA256)
    COMPUTED_CHECKSUM=$(sha256sum "${FILE}" | cut -d\  -f1)
    ;;
  SHA512)
    COMPUTED_CHECKSUM=$(sha512sum "${FILE}" | cut -d\  -f1)
    ;;
  *)
    echo "Unsupported checksum algorithm - ${ALG_UPPER}"
    exit 1
esac
if [ "${CHECKSUM}" == "${COMPUTED_CHECKSUM}" ] ; then
  exit 0
else
  exit 1
fi

Example C:\Scripts\tosca_verify_checksum.ps1 script for Windows:

# C:\Scripts\tosca_verify_checksum.ps1

param(
    [Parameter(Mandatory=$True, Position=0, ValueFromPipeline=$false)]
    [System.String]
    $artifactFile,

    [Parameter(Mandatory=$False, Position=1, ValueFromPipeline=$false)]
    [System.String]
    $checksum,

    [Parameter(Mandatory=$False, Position=2, ValueFromPipeline=$false)]
    [System.String]
    $algorithm
)

if(-not [string]::IsNullOrEmpty($checksum) ) {
  return (Get-FileHash -Path $artifactFile -Algorithm $algorithm ).Hash -eq $checksum
}

return $False

Note: It is not necessary to support a default algorithm to this script, as this TOSCA implementation will always supply the algorithm.

TOSCA Normative Node Types

The following TOSCA Normative Node Types are supported:

  • tosca.nodes.Root - the TOSCA Node Type all other TOSCA base Node Types derive from.
  • tosca.nodes.Abstract.Compute - represents an abstract compute resource without any requirements on storage or network resources.
  • tosca.nodes.Compute - one or more real or virtual processors of software applications or services along with other essential local resources. The resources the compute node represents can logically be viewed as a (real or virtual) “server”.
  • tosca.nodes.SoftwareComponent - represents a generic software component that can be managed and run by a TOSCA Compute Node Type.
  • tosca.nodes.WebServer - represents an abstract software component or service that is capable of hosting and providing management operations for one or more WebApplication nodes. In AMP this is an alias for a SoftwareComponent to allow flexible configurations.
  • tosca.nodes.WebApplication - represents a software application that can be managed and run by a TOSCA WebServer node. In AMP the tosca.capabilities.Endpoint is currently not supported.
  • tosca.nodes.DBMS - represents a typical relational, SQL Database Management System software component or service.
  • tosca.nodes.Database - represents a logical database that can be managed and hosted by a TOSCA DBMS node.
  • tosca.nodes.Abstract.Storage - represents an abstract storage resource without any requirements on compute or network resources.
  • tosca.nodes.Storage.ObjectStorage - represents storage that provides the ability to store data as objects (or BLOBs of data) without consideration for the underlying filesystem or devices. In AMP the tosca.capabilities.Endpoint is currently not supported.
  • tosca.nodes.Storage.BlockStorage - represents storage that provides the ability to store data as objects (or BLOBs of data) without consideration for the underlying filesystem or devices. In AMP the tosca.capabilities.Attachment is currently not supported.
  • tosca.nodes.Container.Runtime - represents operating system-level virtualization technology used to run multiple application services on a single Compute host.
  • tosca.nodes.Container.Application - represents an application that requires Container-level virtualization technology.

Cloudsoft AMP and Cloudsoft AMP Types

Existing Cloudsoft AMP SoftwareProcess types follow a different lifecycle and location model to TOSCA; however they can easily be used from within TOSCA by wrapping it in the special type brooklyn.tosca.entity.SoftwareProcessWrapper (which extends tosca.nodes.SoftwareComponent). This will ensure it gets the TOSCA location information, whether from requirements or an implicit tosca.default.location, and that TOSCA lifecycle methods and attributes are exposed by SoftwareProcessWrapper while the wrapped AMP SoftwareProcess can follows its semantics. This can be done as follows(shown for the simple EmptySoftwareProcess, but of course this works for Ansible, Chef, Terraform, and other popular Cloudsoft AMP entities):

topology_template:
  node_templates:
    a_software:
      type: brooklyn.tosca.entity.SoftwareProcessWrapper
      properties:
        wrapped_spec:
          $brooklyn:entitySpec:
            type: org.apache.brooklyn.entity.software.base.EmptySoftwareProcess

AMP Enrichers, Policies and Initializers can be attached to node and topology templates using TOSCA groups. The group used to attach these could be named appropriately, (add_brooklyn_types as shown in the next code YAML sample) and it has to be of type brooklyn.tosca.groups.initializer. The various AMP types are provided as a list, grouped based on their type, as a valued to properties of this type.

tosca_definitions_version: tosca_simple_yaml_1_3

metadata:
  template_name: sample-template
  
topology_template:
  node_templates:
    sample-node:
      type: tosca.nodes.Compute
    
  groups:
  - add_brooklyn_types:
      members: [ < sample-node <or list or memebers> or sample-template>]
      type: brooklyn.tosca.groups.initializer
      properties:
        brooklyn.initializers: <list of AMP Initializers here>
        brooklyn.policies: <list of AMP Policies here>
        brooklyn.enrichers: <list of AMP Enrichers here>
     

TOSCA does not have a definition for a location type. Thus, AMP locations can be used in TOSCA templates, by using groups as well. A location can be set to a node or a topology template; when applied to a topology template, all node templates will be associated with that location. The brooklyn.tosca.groups.initializer is used to attach a location as well as shown in the following example:

tosca_definitions_version: tosca_simple_yaml_1_3

metadata:
  template_name: sample-template
  
topology_template:
  node_templates:
    sample-node:
      type: tosca.nodes.Compute

  groups:
  - add_brooklyn_types:
      members: [ < sample-node <or list or memebers> or sample-template>]
      type: brooklyn.tosca.groups.initializer
      properties:
        location:
        - <locationID>

Re-use of Templates as Node Types

AMP allows for a topology template to be used as a type for node template. This promotes reusability.

Scalar Unit Size

AMP uses the TOSCA specification section 3.3.6.4 definition of unit size for storage types. For example

1 KB = 1 Kilobyte = 1000 bytes 1 KiB = 1 Kibabyte = 1024 bytes