cloudsoft.io

Setting up the QA Framework

Deploying the Components

Deploying the QA Framework

A blueprint for deploying the QA Framework is shipped with AMP in its default catalog.

Deploy an instance of the “amp-qa-framework” to the cloud of your choice. For example:

location:
  ibm-bluebox:
    imageNameRegex: CentOS 7
    loginUser: centos
    minRam: 4G
services:
- type: amp-qa-framework-test
  brooklyn.config:
    couchdb.initial.password: C0uchdb1!
    amp.initial.password: Cl0udsoftamp1!

This will set up three VMs:

  1. AMP (with the QA Entities installed) - this is the QA AMP
  2. Dashboard - to display the test results
  3. CouchDB - to store the test results

Accessing the provisioned VMs:

  • The AMP login will be user “amp” and the given initial password
  • The CouchDB login will be user “qa” and the given initial password

The following requirements apply to these components:

  • CentOS 7
  • Java 8 (recommended is OpenJDK with devel tools)
  • Sufficient linux kernel entropy
  • Minimum 4G RAM
  • Two cores (or equivalent)
  • Minimum 20G disk; more is required for CouchDB (if ‘000s of test runs are to be stored)
  • The following ports must be open:
    • AMP port must be reachable from Jenkins (e.g. port 8443 if using https)
    • CouchDB port 5984 must be reachable from AMP and from the dashboard
    • Dashboard port 8080 must be reachable by QA users
    • AMP target instances must be reachable from AMP (e.g. port 8443 if using https)

Deploying the Target AMP(s)

The Target AMP instances are set up in the normal way - see Get AMP Running in the standard AMP install guide.

Jenkins

This guide assumes that there is a pre-existing Jenkins instance. If there is not, follow the standard Jenkins install guide.

Configuring the QA Entities

Running the “amp-qa-framework” blueprint (above) will have provided you with a QA AMP. When you log in, you will see a running entity called TestScheduler.

You must use Effectors on that entity to configure the AMP instances you want to run your tests against and also the locations where those AMP instances will test your applications.

The Effectors are:

  • newAmpTarget adds a new AMP target instance (along with its access credentials), giving it a name that will subsequently be used in calls to newTest.

    The parameters are:

    • name: the target’s name (e.g. “production-version-amp”)

    • endpoint: the AMP endpoint, including credentials. For example “http://admin:pa55w0rd@10.104.0.2:8081”

  • newLocationTarget adds a new location that can be used as a target for tests.

    The parameters are:

    • name: the target’s name (e.g. “ibm-bluebox-sng-centos7”)

    • catalogItem: the YAML for a location catalog item (with accompanying brooklyn.catalog: and metadata)

    Example Locations (that provide the syntax the catalogItem parameter expects) can be found in the qa-jenkins-seed repo. For example, here is a Google Compute location.

    Note - when adding a location through the AMP UI there is a bug that stops the location being added successfully (the line feeds are removed which results in invalid YAML). To workaround this issue, please use the Classic UI to invoke the newLocationTarget effector. Or invoke it via the CLI.

  • removeAmpTarget will remove an AMP target instance that was previously added via newAmpTarget. It takes a single parameter (name) for the AMP target to be removed. Note that updating an existing AMP target instance endpoint requires removing and re-adding it.

  • removeLocationTarget will remove a location target that was prevoiusly added via newLocationTarget. It takes a single parameter (name) for the location to be removed. Note that updating an existing location requires removing and re-adding it.

Sanity-test of QA Framework

The QA Framework can be easily tested by invoking the newTest effector with a simple blueprint.

The example below assumes that you have an AMP target named “production-version-amp”, a target location named ibm-bluebox-sng-centos7, and the QA entities running on an AMP at http://amp:Cl0udsoftamp1!@10.104.0.1:8081.

This bash does 3 things:

  • creates a YAML file containing the application to be tested
  • creates a YAML file containing the test to check the application is available

  • calls AMP using the CLI to invoke the newTest effector, passing the Target AMP details and the YAML files

    cat <<ENDOFFILE > simple-app.yaml
brooklyn.catalog:
  id: simple-app
  version: 1.0
  itemType: template
  name: Simple app
  item:
    services:
    - type: org.apache.brooklyn.entity.stock.BasicApplication
ENDOFFILE
cat <<ENDOFFILE > simple-app-test.yaml
brooklyn.catalog:
  items:
  - id: simple-app-test
    version: 1.0
    itemType: template
    name: Simple app test
    item:
      services:
      - type: simple-app
        id: targetapp
      - type: org.apache.brooklyn.test.framework.TestSensor
        targetId: targetapp
        sensor: service.isUp
        timeout: 1m
        assert:
        - equals: true
ENDOFFILE
AMP_TARGET="production-version-amp"
LOCATION_TAG="ibm-bluebox-sng-centos7"
br login http://10.104.0.1:8081 amp "Cl0udsoftamp1!"
br app TestScheduler effector newTest invoke \
    -P "ampTargets=${AMP_TARGET}" \
    -P "locationTargets=${LOCATION_TAG}" \
    -P app=@simple-app.yaml \
    -P test=@simple-app-test.yaml

Production Considerations

In a production environment, the following are strongly recommended:

  • The CouchDB database should be regularly backed up. Where high-availability is required, a CouchDB cluster should be used. Note that the CouchDB set up as part of the blueprint “amp-qa-framework” is a single instance. It runs in a Docker container on a dedicated VM, and writes its data to the host’s directory /data/container-storage/couch-db/. See CouchDB documentation for best practices.

  • AMP’s persisted state can be backup up in the usual way, and AMP can optionally be deployed as a high-availability cluster. Note that the AMP set up as part of the blueprint “amp-qa-framework” is a single instance and does not configure any regular backup of persisted state.

  • It is strongly encouraged to store credentials for the location targets in externalised configuration, rather than exposing the credentials as plain text within the location catalog items. If doing this, the target AMP instances must be configured with access to the externalised configuration stores. This is because the location catalog item is added to the target AMP.

  • Use of HTTPS (rather than HTTP) is strongly recommended

  • Use of a non-root user is strongly recommended for all processes

  • Access to the backend services should be locked down (opening ports only to the services that require them)

AMP and Maeztro are developed and supported by Cloudsoft Corporation.