Container Service Locations
Docker Locations
Cloudsoft AMP can deploy applications to:
both provisioned by Cloudsoft AMP and set up manually.
Docker Container Location
Cloudsoft AMP can deploy applications to Docker containers both provisioned by Cloudsoft AMP and set up manually.
Here is an example catalog item to add a Docker engine endpoint to your catalog locations:
brooklyn.catalog:
id: my-docker-engine
name: "My Docker engine"
itemType: location
item:
type: docker
brooklyn.config:
endpoint: https://<< address >>:<< port >>
identity: << path to my cert.pem >>
credential: << path to my key.pem >>
# Default image if no other explicitly set
# imageId: "cloudsoft/centos:7"
Note The endpoint of a Docker engine is the IP + port where the docker engine is currently running. As for the identity and credential, the Docker engine will generate those by default in ~/.docker/certs
folder, unless you specified them during the installation.
Docker Container based blueprints
Once your Docker container location has been configured, AMP can launch instances based on a DockerContainer
entity, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:
# see above for a definition of the location
location: my-docker-engine
services:
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress-mysql
name: MySQL
brooklyn.config:
mysql.root_password: password
docker.container.imageName: mysql:5.6
# Maps the port to the host node, making it available for external access
docker.container.inboundPorts:
- "3306"
docker.container.environment:
MYSQL_ROOT_PASSWORD: $brooklyn:config("mysql.root_password")
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress
name: Wordpress
brooklyn.config:
docker.container.imageName: wordpress:4-apache
# Maps the port to the host node, making it available for external access
docker.container.inboundPorts:
- "80"
docker.container.environment:
WORDPRESS_DB_HOST: $brooklyn:entity("wordpress-mysql").attributeWhenReady("host.subnet.address")
WORDPRESS_DB_PASSWORD: $brooklyn:entity("wordpress-mysql").config("mysql.root_password")
Docker container configuration
To configure the DockerContainer
entity, the following configuration params are available:
- docker.container.disableSsh Skip checks such as ssh for when docker image doesn’t allow ssh; use the default image
cloudsoft/centos:7
for ssh-able image - docker.container.imageName Image name to pull from docker hub; overrides the default one
cloudsoft/centos:7
- docker.container.inboundPorts List of ports, that the docker image maps to the host, opening them to the public
- docker.container.environment Environment variables to set on container startup; this must be a map
Docker Swarm Location
Cloudsoft AMP can deploy applications to Docker Swarms both provisioned by Cloudsoft AMP and set up manually.
Here is an example catalog item to add a Docker Swarm endpoint to your catalog locations:
brooklyn.catalog:
id: my-docker-swarm
name: "My Docker Swarm"
itemType: location
item:
type: docker
brooklyn.config:
endpoint: https://<< address >>:<< port >>
identity: << path to my cert.pem >>
credential: << path to my key.pem >>
# Default image if no other explicitly set
# imageId: "cloudsoft/centos:7"
templateOptions:
networkMode: "brooklyn"
Note if you have provisioned your own docker swarm you may need to first pull the Cloudsoft configured image on the Swarm Manager. Another recommended step is to create a default network for the containers:
docker -H ${swarm_endpoint} ${TLS_OPTIONS} pull cloudsoft/centos:7
docker -H ${swarm_endpoint} ${TLS_OPTIONS} images --no-trunc
docker network create --driver=overlay brooklyn
Credentials for Deploying to Docker Swarm
To deploy to a Docker Swarm endpoint, you’ll need pem files for identity/credential. These can either be copied from one of the Docker Engine VMs, or can be generated locally and signed by the certificate authority. The actual IP of the client doesn’t matter.
To generate your own certificates and signed them with the example CA server included in AMP (note this is not recommended for use in a production environment and could be subject to future removal):
# Create your certificates directory
mkdir -p .certs
# Get yourself a certificate from the CA
# You can use any IP; to find your IP use `ifconfig`
own_ip=192.168.1.64
ca=$(br app "Docker Swarm" ent ca-server sensor main.uri)
echo ${ca}
curl -L ${ca}/cacert/ca.pem --output .certs/ca.pem
openssl genrsa -out .certs/key.pem 2048
openssl req -new -key .certs/key.pem -days 1825 -out .certs/csr.pem -subj "/CN=${own_ip}"
curl -X POST --data-binary @.certs/csr.pem ${ca}/sign > .certs/cert.pem
To be able to execute docker ...
commands locally:
# Set up TLS options to point at your certificates (created above)
CERTS_DIR=$(pwd)/.certs
TLS_OPTIONS="--tlsverify --tlscacert=${CERTS_DIR}/ca.pem --tlscert=${CERTS_DIR}/cert.pem --tlskey=${CERTS_DIR}/key.pem"
# Check docker works
swarm_endpoint=$(br app "Docker Swarm" ent "swarm-cluster" sensor swarm.url)
echo ${swarm_endpoint}
docker -H ${swarm_endpoint} ${TLS_OPTIONS} ps
# Run something, and check it is listed
docker -H ${swarm_endpoint} ${TLS_OPTIONS} run hello-world
docker -H ${swarm_endpoint} ${TLS_OPTIONS} ps -a
Instead of explicit parameters to docker
you can use its environment variables as follows:
export DOCKER_HOST=tcp://10.10.10.152:3376
export DOCKER_TLS_VERIFY=true
export DOCKER_CERT_PATH=$(pwd)/.certs
docker ps -a
Kubernetes Location
Cloudsoft AMP can deploy applications to Kubernetes (k8s) clusters both provisioned by Cloudsoft AMP and set up manually.
Here is an example catalog item to add a Kubernetes endpoint to your catalog locations:
brooklyn.catalog:
id: my-kubernetes-cluster
name: "My Kubernetes Cluster"
itemType: location
item:
type: kubernetes
brooklyn.config:
endpoint: << endpoint >>
identity: "guest"
credential: "guest"
image: "cloudsoft/centos:7"
loginUser.password: "p4ssw0rd"
There are a lot of ways to authenticate with kubernetes. AMP configuration for these are documented in the reference. For example to use client certificates use the following example yaml:
brooklyn.catalog:
id: my-kubernetes-cluster
name: "My Kubernetes Cluster"
itemType: location
item:
type: kubernetes
brooklyn.config:
endpoint: << endpoint >>
caCertData: |
<< Generated Ca Cert (see below) >>
clientCertData: |
<< Generated Cert (see below) >>
clientKeyData: |
<< Generated client key (see below) >>
image: "cloudsoft/centos:7"
loginUser.password: "p4ssw0rd"
AMP Deploys to a Kubernetes cluster by modelling a KubernetesPod
entity which is made up of multiple heterogeneous DockerContainer
entities.
Plain-blueprints
Standard blueprints can be deployed within a K8s cluster, here’s a simple example:
location:
<< see above >>
services:
- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess
name: "Simple Netcat Server"
brooklyn.config:
provisioning.properties:
inboundPorts: [ 22, 4321 ]
env:
CLOUDSOFT_ROOT_PASSWORD: "p4ssw0rd"
launch.command: |
yum install -y nc
echo hello | nc -l 4321 &
echo $! > $PID_FILE
For each entity AMP will create a Deployment.
This deployment contains a ReplicaSet
of replicas (defaulting to one) of a Pod.
Each pod contains a single SSHable container based on the cloudsoft/centos:7
image.
It will then install and launch the entity. Each inboundPort
will be exposed as a
NodePort in a Service.
The config options in the provisioning.properties
section allow the location to be further customized for each entity, as follows:
- env The
cloudsoft/centos:7
image uses an environment variable namedCLOUDSOFT_ROOT_PASSWORD
to assign the SSH login user password. This must match theloginUser.password
configuration on the location. - inboundPorts The set of ports that should be exposed by the service.
Docker Container based blueprints
Alternatively AMP can launch instances based on a DockerContainer
, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:
location:
<< see above >>
services:
- type: io.cloudsoft.amp.containerservice.kubernetes.entity.KubernetesPod
brooklyn.children:
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress-mysql
name: "MySQL"
brooklyn.config:
docker.container.imageName: mysql:5.6
docker.container.inboundPorts: [ "3306" ]
docker.container.environment:
MYSQL_ROOT_PASSWORD: "password"
provisioning.properties:
deployment: wordpress-mysql
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress
name: "Wordpress"
brooklyn.config:
docker.container.imageName: wordpress:4-apache
docker.container.inboundPorts: [ "80" ]
docker.container.environment:
WORDPRESS_DB_HOST: "wordpress-mysql"
WORDPRESS_DB_PASSWORD: "password"
The DockerContainer
entities each create their own DeploymentConfig, ReplicationController and Pod entities,
in the same way as the standard AMP blueprint entities above. Each container entity can be further configured using the following options:
- docker.container.imageName The Docker image to use for the container
- docker.container.inboundPorts The set of ports on the container that should be exposed
- docker.container.environment A map of environment variables for the container
Note the use of deployment in the provisioning.properties
configuration, to set the hostname of the MySQL container to allow the Wordpress Apache server to connect to it.
Kubernetes location configuration
To configure the kubernetes location for different kubernetes setups the following configuration params are available.
- caCertData Data for CA certificate
- caCertFile URL of resource containing CA certificate data
- clientCertData Data for client certificate
- clientCertFile URL of resource containing client certificate data
- clientKeyData Data for client key
- clientKeyFile URL of resource containing client key data
- clientKeyAlgo Algorithm used for the client key
- clientKeyPassphrase Passphrase used for the client key
- oauthToken The OAuth token data for the current user
- namespace Namespace where resources will live; the default is ‘amp’
- namespace.create Whether to create the namespace if it does not exist
- default true
- namespace.deleteEmpty Whether to delete an empty namespace when releasing resources
- default false
- persistentVolumes Set up persistent volumes.
- deployment The name of the service the deployment will use.
- image Docker image to be deployed into the pod
- osFamily OS family, e.g. CentOS, Ubuntu
- osVersionRegex Regular expression for the OS version to load
- env Environment variables to inject when starting a container
- replicas Number of replicas of the pod
- default 1
- secrets Kubernetes secrets to be added to the pod
- limits Kubernetes resource limits
- privileged Whether Kubernetes should allow privileged containers
- default false
- loginUser Override the user who logs in initially to perform setup
- default root
- loginUser.password Custom password for the user who logs in initially
- injectLoginCredential Whether to inject login credentials (if null, will infer from image choice); ignored if explicit ‘loginUser.password’ supplied
OpenShift Location
Cloudsoft AMP can deploy applications to Red Hat OpenShift clusters.
Here is an example catalog item to add an OpenShift endpoint to your catalog locations:
brooklyn.catalog:
id: my-openshift-cluster
name: "My Openshift Cluster"
itemType: location
item:
type: openshift
brooklyn.config:
endpoint: << endpoint >>
caCertData: |
<< Generated Ca Cert (see below) >>
clientCertData: |
<< Generated Cert (see below) >>
clientKeyData: |
<< Generated client key (see below) >>
namespace: << project name >>
privileged: true
- Endpoint
The endpoint key is the https URL of your OpenShift master. AMP connects to this to provision applications on the cluster.
- OpenShift Authorization
The caCertData
, clientCertData
and clientKeyData
are the credentials for your OpenShift cluster. Note that
they can also be given as paths to files using the keys caCertFile
, clientCertFile
and clientKeyFile
. See the
OpenShift documentation for
more detail on the content of these.
An alternate way of authorizing your OpenShift is using OAuth. To obtain the token required you must use the oc
command-line tool,
first to log in to OpenShift and then to display the token value, using the whoami
command:
oc login << endpoint >>
oc whoami -t
Which will output the token to the command line:
mzUTj0JmWDYLSspumvW5B74rn8geKd6Qll11IPkaqeE
This is then set as the oauthToken
field in the location:
brooklyn.catalog:
id: my-openshift-cluster
name: "My Openshift Cluster"
itemType: location
item:
type: openshift
brooklyn.config:
endpoint: << endpoint >>
oauthToken: mzUTj0JmWDYLSspumvW5B74rn8geKd6Qll11IPkaqeE
namespace: << project name >>
privileged: true
- Namespace
The namespace
key relates to the project in which your AMP managed applications will deploy. If no project exists,
you will first need to log into your OpenShift cluster and create a project. The namespace
key should then contain
the ID of this.
OpenShift Configuration
AMP requires that you configure your OpenShift instance with the following options to allow it to fully provision and manage applications.
- Container Privileges
Depending on how the images you wish to use have been created, you may need to set up accounts and permissions to allow them to run.
Containers written for the OpenShift platform follow certain rules such as logging to the console to allow centralized log
management or avoiding the root
user since the platform will use an arbitrary user id. For applications that follow these rules
the default restricted
security constraints are all that is needed. When using images from Docker Hub, or the cloudsoft/centos:7
image used by native AMP entities, privileged access must be enabled. This can be done by creating a new user for your application,
and assigning it the privileged
or anyuid
security constraints as described in the documentation.
Alternatively, for development systems where security is not an issue, you can edit the restricted
constraint directly, and
set the configuration option allowPrivilegedContainer
to true
and runAsUser
to have type RunAsAny
. This can be configured
using the oc command to edit the cluster configuration:
oc login << endpoint >>
sudo oc edit scc restricted
Plain-blueprints
Standard blueprints can be deployed within an OpenShift cluster, here’s a simple example:
location:
<< see above >>
services:
- type: org.apache.brooklyn.entity.software.base.VanillaSoftwareProcess
name: "Simple Netcat Server"
brooklyn.config:
provisioning.properties:
inboundPorts: [ 22, 4321 ]
env:
CLOUDSOFT_ROOT_PASSWORD: "p4ssw0rd"
launch.command: |
yum install -y nc
echo hello | nc -l 4321 &
echo $! > $PID_FILE
For each entity AMP will create a DeploymentConfig.
This deployment configuration contains a ReplicationController
with replicas (defaulting to one) of a Pod.
Each pod contains a single SSHable container based on the cloudsoft/centos:7
image.
It will then install and launch the entity. Each inboundPort
will be exposed as a
NodePort in a Service.
The config options in the provisioning.properties
section allow the location to be further customized for each entity, as follows:
- env The
cloudsoft/centos:7
image uses an environment variable namedCLOUDSOFT_ROOT_PASSWORD
to assign the SSH login user password. This must match theloginUser.password
configuration on the location. - inboundPorts The set of ports that should be exposed by the service.
Note the use of deployment in the provisioning.properties
configuration, to set the hostname of the MySQL container to allow the Wordpress Apache server to connect to it.
DockerContainer based blueprints
Alternatively AMP can launch instances based on a DockerContainer
, this means additional configuration such as custom docker images can be specified. Here’s an example which sets up a Wordpress instance:
location:
<< see above >>
services:
- type: io.cloudsoft.amp.containerservice.kubernetes.entity.KubernetesPod
brooklyn.children:
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress-mysql
name: "MySQL"
brooklyn.config:
docker.container.imageName: mysql:5.6
docker.container.inboundPorts: [ "3306" ]
docker.container.environment:
MYSQL_ROOT_PASSWORD: "password"
provisioning.properties:
deployment: wordpress-mysql
- type: io.cloudsoft.amp.containerservice.dockercontainer.DockerContainer
id: wordpress
name: "Wordpress"
brooklyn.config:
docker.container.imageName: wordpress:4-apache
docker.container.inboundPorts: [ "80" ]
docker.container.environment:
WORDPRESS_DB_HOST: "wordpress-mysql"
WORDPRESS_DB_PASSWORD: "password"
The DockerContainer
entities each create their own DeploymentConfig, ReplicationController and Pod entities,
in the same way as the standard AMP blueprint entities above. Each container entity can be further configured using the following options:
- docker.container.imageName The Docker image to use for the container
- docker.container.inboundPorts The set of ports on the container that should be exposed
- docker.container.environment A map of environment variables for the container
OpenShift location configuration
The OpenShift location uses the same configuration options as the Kubernetes location, with the following exception:
- namespace Also refers to the OpenShift project the Pod will be started in.