Architecture

The architecture of a standard QA Test Framework configuration is shown below:

In brief, the components are:

• Jenkins is used to schedule tests. For example, blueprints could be tested nightly, or triggered by a pull request or commit.

• The “QA Entities” coordinate the tests. This runs as an application inside an AMP instance. When triggered via an effector call, the new test is run and the results collected. The top-level qa entity is called the “TestScheduler”, but this is not to be confused with Jenkin’s role of scheduling and triggering jobs.

• The “Target AMP” is the desired AMP on which the test blueprint will be run (i.e. the desired version of AMP on the desired platform). The test blueprint is deployed and execute there, along with the test assertions. The “QA Entities” are configured with the URLs of one or more target AMPs.

• CouchDB is the database used for storing the test results.

• The Dashboard is the web-based user interface for viewing test results.

The process flow shown above can be described as:

• Jenkins invokes the newTest effector on the top-level QA Entity (e.g. using the br CLI tool). It passes the catalog item(s) to be added, the catalog item for the test, the desired location, and the AMP to target. The effector result indicates success or fail.

• The QA Entity executes the blueprint on the target AMP, using its standard REST API. This involves:

  • Identifies the target AMP instance, and the target location.
  • Adds the catalog items to the target AMP
  • Adds the location to the target AMP’s catalog
  • Deploys the blueprint
  • Waits for the blueprint instance to finish (or to fail)
  • Retrieves the test results.
  • Tears down the blueprint instance
  • Remove the catalog items (including location) from the target AMP