Container Service Locations

Docker Locations

Cloudsoft AMP can deploy applications to:

• Docker containers
• Docker Swarms
• Kubernetes Clusters
• Openshift Clusters

both provisioned by Cloudsoft AMP and set up manually.

Docker Container Location

Cloudsoft AMP can deploy applications to Docker containers both provisioned by Cloudsoft AMP and set up manually.

Here is an example catalog item to add a Docker engine endpoint to your catalog locations:

brooklyn.catalog:
  id: my-docker-engine
  name: "My Docker engine"
  itemType: location
  item:
    type: docker
    brooklyn.config:
      endpoint: https://<< address >>:<< port >>
      identity: << path to my cert.pem >>
      credential: << path to my key.pem >>
      # Default image if no other explicitly set
      # imageId: "cloudsoft/centos:7"

Note The endpoint of a Docker engine is the IP + port where the docker engine is currently running. As for the identity and credential, the Docker engine will generate those by default in ~/.docker/certs folder, unless you specified them during the installation.

Docker Container based blueprints

Once your Docker container location has been configured, AMP can launch instances based on a DockerContainer entity, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:

# see above for a definition of the location
location: my-docker-engine

services:
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
  id: wordpress-mysql
  name: MySQL
  brooklyn.config:
    mysql.root_password: password
    docker.container.imageName: mysql:5.6
    # Maps the port to the host node, making it available for external access
    docker.container.inboundPorts:
    - "3306"
    docker.container.environment: 
      MYSQL_ROOT_PASSWORD: $brooklyn:config("mysql.root_password")
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
  id: wordpress
  name: Wordpress
  brooklyn.config:
    docker.container.imageName: wordpress:4-apache
    # Maps the port to the host node, making it available for external access
    docker.container.inboundPorts:
    - "80"
    docker.container.environment: 
      WORDPRESS_DB_HOST: $brooklyn:entity("wordpress-mysql").attributeWhenReady("host.subnet.address")
      WORDPRESS_DB_PASSWORD: $brooklyn:entity("wordpress-mysql").config("mysql.root_password")

Docker container configuration

To configure the DockerContainer entity, the following configuration params are available:

• docker.container.disableSsh Skip checks such as ssh for when docker image doesn’t allow ssh; use the default image cloudsoft/centos:7 for ssh-able image
• docker.container.imageName Image name to pull from docker hub; overrides the default one cloudsoft/centos:7
• docker.container.inboundPorts List of ports, that the docker image maps to the host, opening them to the public
• docker.container.environment Environment variables to set on container startup; this must be a map

Docker Swarm Location

Cloudsoft AMP can deploy applications to Docker Swarms both provisioned by Cloudsoft AMP and set up manually.

Here is an example catalog item to add a Docker Swarm endpoint to your catalog locations:

brooklyn.catalog:
  id: my-docker-swarm
  name: "My Docker Swarm"
  itemType: location
  item:
    type: docker
    brooklyn.config:
      endpoint: https://<< address >>:<< port >>
      identity: << path to my cert.pem >>
      credential: << path to my key.pem >>
      # Default image if no other explicitly set
      # imageId: "cloudsoft/centos:7"
      templateOptions:
        networkMode: "brooklyn"

Note if you have provisioned your own docker swarm you may need to first pull the Cloudsoft configured image on the Swarm Manager. Another recommended step is to create a default network for the containers:

docker -H ${swarm_endpoint}  ${TLS_OPTIONS} pull cloudsoft/centos:7
docker -H ${swarm_endpoint}  ${TLS_OPTIONS} images --no-trunc
docker network create --driver=overlay brooklyn

Credentials for Deploying to Docker Swarm

To deploy to a Docker Swarm endpoint, you’ll need pem files for identity/credential. These can either be copied from one of the Docker Engine VMs, or can be generated locally and signed by the certificate authority. The actual IP of the client doesn’t matter.

To generate your own certificates and signed them with the example CA server included in AMP (note this is not recommended for use in a production environment and could be subject to future removal):

# Create your certificates directory
mkdir -p .certs

# Get yourself a certificate from the CA
# You can use any IP; to find your IP use `ifconfig`
own_ip=192.168.1.64
ca=$(br app "Docker Swarm" ent ca-server sensor main.uri)
echo ${ca}
curl -L ${ca}/cacert/ca.pem --output .certs/ca.pem
openssl genrsa -out .certs/key.pem 2048
openssl req  -new -key .certs/key.pem -days 1825 -out .certs/csr.pem -subj "/CN=${own_ip}"
curl -X POST --data-binary @.certs/csr.pem ${ca}/sign > .certs/cert.pem

To be able to execute docker ... commands locally:

# Set up TLS options to point at your certificates (created above)
CERTS_DIR=$(pwd)/.certs
TLS_OPTIONS="--tlsverify --tlscacert=${CERTS_DIR}/ca.pem --tlscert=${CERTS_DIR}/cert.pem --tlskey=${CERTS_DIR}/key.pem"

# Check docker works
swarm_endpoint=$(br app "Docker Swarm" ent "swarm-cluster" sensor swarm.url)
echo ${swarm_endpoint}
docker -H ${swarm_endpoint} ${TLS_OPTIONS} ps

# Run something, and check it is listed
docker -H ${swarm_endpoint} ${TLS_OPTIONS} run hello-world
docker -H ${swarm_endpoint} ${TLS_OPTIONS} ps -a

Instead of explicit parameters to docker you can use its environment variables as follows:

export DOCKER_HOST=tcp://10.10.10.152:3376
export DOCKER_TLS_VERIFY=true
export DOCKER_CERT_PATH=$(pwd)/.certs
docker ps -a

Kubernetes Location

Cloudsoft AMP can deploy applications to Kubernetes (k8s) clusters both provisioned by Cloudsoft AMP and set up manually.

Here is an example catalog item to add a Kubernetes endpoint to your catalog locations:

brooklyn.catalog:
  id: my-kubernetes-cluster
  name: "My Kubernetes Cluster"
  itemType: location
  item:
    type: kubernetes
    brooklyn.config:
      endpoint: << endpoint >>
      identity: "guest"
      credential: "guest"
      image: "cloudsoft/centos:7"
      loginUser.password: "p4ssw0rd"

There are a lot of ways to authenticate with kubernetes. AMP configuration for these are documented in the reference. For example to use client certificates use the following example yaml:

brooklyn.catalog:
  id: my-kubernetes-cluster
  name: "My Kubernetes Cluster"
  itemType: location
  item:
    type: kubernetes
    brooklyn.config:
      endpoint: << endpoint >>
      caCertData: |
        << Generated Ca Cert (see below) >>
      clientCertData: |
        << Generated Cert (see below) >>
      clientKeyData: |
        << Generated client key (see below) >>
      image: "cloudsoft/centos:7"
      loginUser.password: "p4ssw0rd"

AMP Deploys to a Kubernetes cluster by modelling a KubernetesPod entity which is made up of multiple heterogeneous DockerContainer entities.

Plain-blueprints

Standard blueprints can be deployed within a K8s cluster, here’s a simple example:

location:
  << see above >>

services:
  - type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess
    name: "Simple Netcat Server"
    brooklyn.config:
      provisioning.properties:
        inboundPorts: [ 22, 4321 ]
        env:
          CLOUDSOFT_ROOT_PASSWORD: "p4ssw0rd"
      launch.command: |
        yum install -y nc
        echo hello | nc -l 4321 &
        echo $! > $PID_FILE

For each entity AMP will create a Deployment. This deployment contains a ReplicaSet of replicas (defaulting to one) of a Pod. Each pod contains a single SSHable container based on the cloudsoft/centos:7 image.

It will then install and launch the entity. Each inboundPort will be exposed as a NodePort in a Service.

The config options in the provisioning.properties section allow the location to be further customized for each entity, as follows:

• env The cloudsoft/centos:7 image uses an environment variable named CLOUDSOFT_ROOT_PASSWORD to assign the SSH login user password. This must match the loginUser.password configuration on the location.
• inboundPorts The set of ports that should be exposed by the service.

Docker Container based blueprints

Alternatively AMP can launch instances based on a DockerContainer, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:

location:
  << see above >>

services:
- type: io.cloudsoft.amp.containerservice.kubernetes.entity.KubernetesPod
  brooklyn.children:
  - type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
    id: wordpress-mysql
    name: "MySQL"
    brooklyn.config:
      docker.container.imageName: mysql:5.6
      docker.container.inboundPorts: [ "3306" ]
      docker.container.environment:
        MYSQL_ROOT_PASSWORD: "password"
      provisioning.properties:
        deployment: wordpress-mysql
  - type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
    id: wordpress
    name: "Wordpress"
    brooklyn.config:
      docker.container.imageName: wordpress:4-apache
      docker.container.inboundPorts: [ "80" ]
      docker.container.environment:
        WORDPRESS_DB_HOST: "wordpress-mysql"
        WORDPRESS_DB_PASSWORD: "password"

The DockerContainer entities each create their own DeploymentConfig, ReplicationController and Pod entities, in the same way as the standard AMP blueprint entities above. Each container entity can be further configured using the following options:

• docker.container.imageName The Docker image to use for the container
• docker.container.inboundPorts The set of ports on the container that should be exposed
• docker.container.environment A map of environment variables for the container

Note the use of deployment in the provisioning.properties configuration, to set the hostname of the MySQL container to allow the Wordpress Apache server to connect to it.

Kubernetes location configuration

To configure the kubernetes location for different kubernetes setups the following configuration params are available.

• caCertData Data for CA certificate
• caCertFile URL of resource containing CA certificate data
• clientCertData Data for client certificate
• clientCertFile URL of resource containing client certificate data
• clientKeyData Data for client key
• clientKeyFile URL of resource containing client key data
• clientKeyAlgo Algorithm used for the client key
• clientKeyPassphrase Passphrase used for the client key
• oauthToken The OAuth token data for the current user
• namespace Namespace where resources will live; the default is ‘amp’
• namespace.create Whether to create the namespace if it does not exist
  • default true
• namespace.deleteEmpty Whether to delete an empty namespace when releasing resources
  • default false
• persistentVolumes Set up persistent volumes.
• deployment The name of the service the deployment will use.
• image Docker image to be deployed into the pod
• osFamily OS family, e.g. CentOS, Ubuntu
• osVersionRegex Regular expression for the OS version to load
• env Environment variables to inject when starting a container
• replicas Number of replicas of the pod
  • default 1
• secrets Kubernetes secrets to be added to the pod
• limits Kubernetes resource limits
• privileged Whether Kubernetes should allow privileged containers
  • default false
• loginUser Override the user who logs in initially to perform setup
  • default root
• loginUser.password Custom password for the user who logs in initially
• injectLoginCredential Whether to inject login credentials (if null, will infer from image choice); ignored if explicit ‘loginUser.password’ supplied

OpenShift Location

Cloudsoft AMP can deploy applications to Red Hat OpenShift clusters.

Here is an example catalog item to add an OpenShift endpoint to your catalog locations:

brooklyn.catalog:
  id: my-openshift-cluster
  name: "My Openshift Cluster"
  itemType: location
  item:
    type: openshift
    brooklyn.config:
      endpoint: << endpoint >>
      caCertData: |
        << Generated Ca Cert (see below) >>
      clientCertData: |
        << Generated Cert (see below) >>
      clientKeyData: |
        << Generated client key (see below) >>
      namespace: << project name >>
      privileged: true

• Endpoint

The endpoint key is the https URL of your OpenShift master. AMP connects to this to provision applications on the cluster.

• OpenShift Authorization

The caCertData, clientCertData and clientKeyData are the credentials for your OpenShift cluster. Note that they can also be given as paths to files using the keys caCertFile, clientCertFile and clientKeyFile. See the OpenShift documentation for more detail on the content of these.

An alternate way of authorizing your OpenShift is using OAuth. To obtain the token required you must use the oc command-line tool, first to log in to OpenShift and then to display the token value, using the whoami command:

oc login << endpoint >>
oc whoami -t

Which will output the token to the command line:

mzUTj0JmWDYLSspumvW5B74rn8geKd6Qll11IPkaqeE

This is then set as the oauthToken field in the location:

brooklyn.catalog:
  id: my-openshift-cluster
  name: "My Openshift Cluster"
  itemType: location
  item:
    type: openshift
    brooklyn.config:
      endpoint: << endpoint >>
      oauthToken: mzUTj0JmWDYLSspumvW5B74rn8geKd6Qll11IPkaqeE
      namespace: << project name >>
      privileged: true

• Namespace

The namespace key relates to the project in which your AMP managed applications will deploy. If no project exists, you will first need to log into your OpenShift cluster and create a project. The namespace key should then contain the ID of this.

OpenShift Configuration

AMP requires that you configure your OpenShift instance with the following options to allow it to fully provision and manage applications.

• Container Privileges

Depending on how the images you wish to use have been created, you may need to set up accounts and permissions to allow them to run. Containers written for the OpenShift platform follow certain rules such as logging to the console to allow centralized log management or avoiding the root user since the platform will use an arbitrary user id. For applications that follow these rules the default restricted security constraints are all that is needed. When using images from Docker Hub, or the cloudsoft/centos:7 image used by native AMP entities, privileged access must be enabled. This can be done by creating a new user for your application, and assigning it the privileged or anyuid security constraints as described in the documentation.

Alternatively, for development systems where security is not an issue, you can edit the restricted constraint directly, and set the configuration option allowPrivilegedContainer to true and runAsUser to have type RunAsAny. This can be configured using the oc command to edit the cluster configuration:

oc login << endpoint >>
sudo oc edit scc restricted

Plain-blueprints

Standard blueprints can be deployed within an OpenShift cluster, here’s a simple example:

location:
  << see above >>

services:
  - type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess
    name: "Simple Netcat Server"
    brooklyn.config:
      provisioning.properties:
        inboundPorts: [ 22, 4321 ]
        env:
          CLOUDSOFT_ROOT_PASSWORD: "p4ssw0rd"
      launch.command: |
        yum install -y nc
        echo hello | nc -l 4321 &
        echo $! > $PID_FILE

For each entity AMP will create a DeploymentConfig. This deployment configuration contains a ReplicationController with replicas (defaulting to one) of a Pod. Each pod contains a single SSHable container based on the cloudsoft/centos:7 image.

It will then install and launch the entity. Each inboundPort will be exposed as a NodePort in a Service.

The config options in the provisioning.properties section allow the location to be further customized for each entity, as follows:

• env The cloudsoft/centos:7 image uses an environment variable named CLOUDSOFT_ROOT_PASSWORD to assign the SSH login user password. This must match the loginUser.password configuration on the location.
• inboundPorts The set of ports that should be exposed by the service.

Note the use of deployment in the provisioning.properties configuration, to set the hostname of the MySQL container to allow the Wordpress Apache server to connect to it.

DockerContainer based blueprints

Alternatively AMP can launch instances based on a DockerContainer, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:

location:
  << see above >>

services:
  - type: io.cloudsoft.amp.containerservice.kubernetes.entity.KubernetesPod
    brooklyn.children:
      - type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
        id: wordpress-mysql
        name: "MySQL"
        brooklyn.config:
          docker.container.imageName: mysql:5.6
          docker.container.inboundPorts: [ "3306" ]
          docker.container.environment:
            MYSQL_ROOT_PASSWORD: "password"
          provisioning.properties:
            deployment: wordpress-mysql
      - type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
        id: wordpress
        name: "Wordpress"
        brooklyn.config:
          docker.container.imageName: wordpress:4-apache
          docker.container.inboundPorts: [ "80" ]
          docker.container.environment:
            WORDPRESS_DB_HOST: "wordpress-mysql"
            WORDPRESS_DB_PASSWORD: "password"

The DockerContainer entities each create their own DeploymentConfig, ReplicationController and Pod entities, in the same way as the standard AMP blueprint entities above. Each container entity can be further configured using the following options:

• docker.container.imageName The Docker image to use for the container
• docker.container.inboundPorts The set of ports on the container that should be exposed
• docker.container.environment A map of environment variables for the container

OpenShift location configuration

The OpenShift location uses the same configuration options as the Kubernetes location, with the following exception:

• namespace Also refers to the OpenShift project the Pod will be started in.