cloudsoft.io

Jenkins Configuration

Jenkins is configured with a set of “jobs”, which can then be executed. A job can be a “matrix” of sub-jobs, for example to target different AMP versions and different cloud providers.

In brief, the Jenkins job will retrieve the blueprint to be tested (e.g. from a Github repo), and make Effector calls to the AMP instance (using the br CLI tool). The technicalities of how Jenkins jobs are configured and orchestrated will be unimportant to most users. Further details on this are provided in later sections of this guide.

The QA Framework uses the Jenkins DSL plug-in to auto-generate these jobs.

Jenkins Prerequisites

The following Jenkins Plugins are required:

  • credentials-binding
  • credentials
  • custom-tools-plugin
  • build-publisher
  • job-dsl
  • matrix-project
  • git
  • Git PR Builder
  • Config File Provider Plugin

Please note there has been a change in Jeknins 2.x to how parameters are handled see Jenkins Issue #34758.

Add the following java opt: -Dhudson.model.ParametersAction.keepUndefinedParameters=true

Auto-generating Jenkins Jobs

The recommended approach to auto-generate the Jenkins Jobs is based on a Git repo (e.g. containing the list of tests and required scripts), and a “Seed Job” that reads this. We provide a starter pack for you which is available on Github:

To create the Git repo, fork the Github repo https://github.com/cloudsoft/qa-jenkins-seed/.

List of Tests

The blueprint to be tested are defined in a JSON file, which references the Git repos that contain the actual blueprints.

Each entry in the tests.json file will seed a blueprint QA job. For example:

[
  {
    "author": "brooklyncentral",
    "name": "brooklyn-tomcat-7-server",
    "branch": "master",
    "repoName": "brooklyncentral/brooklyn-tomcat-7-server",
    "sources" : ["catalog.bom"],
    "tests" : ["catalog.tests.bom"],
    "additionalLocations" : {},
    "brooklynTags": ["latest", "AMP-3.2.0"],
    "locationTags": ["bluebox-london-centos"],
    "triggers": ["pr", "nightly"]
  }
]

The fields are:

Field Type Description
author String The author of the repository
name String The name of the repository
branch String The branch to test
repoName String The uri of the repository
sources Array The path to the yaml catalog items (within the repo), to be temporarily added to the AMP target. If this lists multiple .bom files, these will be automatically merged into a single file, which is then added to the target AMP when running the test.
tests Array The path to the yaml test catalog items (within the repo), to be temporarily added to the AMP target
additionalLocations Array Path to yaml catalog items (within the repo) for additional locations to be added
brooklynTags Array The names of the AMP targets to test against. This must match the names used when calling newAmpTarget.
locationTags Array The names of the location targets to test against. This must match the names used when calling newLocationTarget.
triggers Array When to automatically run the test (e.g. “nightly” or “pr”). The “nightly” trigger will mean jobs are run automatically every 24 hours. The “pr” trigger will run when a PR is opened.

Jenkins Seed Job

Use the Jenkins seed job to auto-generate (or re-generate) all the other jobs.

As mentioned above, Cloudsoft provide an example seed job in the qa-jenkins-seed repo which you can fork.

After doing that you must configure your Jenkins so it can use the seed job successfully. There are three configuration steps:

Global Tool Configuration

In your Jenkins you will need to configure the tools for the seed job to use (if they are not already configured).

  • Do this under ‘Manage Jenkins’ –> ‘Global Tool Configuration’.
Tool Description
Maven Name as ‘maven’ and configure to install automatically
Groovy Name as ‘groovy’ and configure to install automatically
Docker Name as ‘docker’ and configure to install automatically
Custom Tool Add ‘brooklyn-cli’ and configure to install automatically (using brooklyn-cli bash command below). Set the Tool Home to ./

brooklyn-cli bash command:

  
#!/usr/bin/env bash
if [ ! -f br ]; then
   wget https://artifactory.cloudsoftcorp.com/artifactory/libs-release-local/org/apache/brooklyn/brooklyn-client-cli/0.10.0-20161108.1349/brooklyn-client-cli-0.10.0-20161108.1349-bin.zip
    unzip brooklyn-client-cli-0.10.0-20161108.1349-bin.zip
    mv brooklyn-client-cli-0.10.0-20161108.1349/linux.386/br ./
    rm -rf brooklyn-client-cli-0.10.0-20161108.1349-bin.zip brooklyn-client-cli-0.10.0-20161108.1349
fi

Configure Jenkins with credentials

You will need to configure the credentials so your Jenkins can access the QA AMP and your repos (where your application blueprints and tests are).

  • Do this under ‘Jenkins’ –> ‘Credentials’:
ID Kind Description
github-credentials Username with password Git credentials
amp-qa-credentials Username with password AMP QA server credentials

Add master seed job

In Jenkins, create a new Freestyle project. The Freestyle project should checkout the master branch of this repository using the github-credentials credentials configured above.

Parameters

  • Select the This project is parameterized option and add the following:
Parameter Name Guidance on Default Value Type
BK_URI The URI of the AMP with the QA Entities String
BK_SCHEDULER_APP_ID Add ‘TestScheduler’ String
BK_ADMIN_CREDENTIALS Set the DefaultValue to the use the QA credentials you added above Credentials
GIT_CREDENTIALS_ID Set the DefaultValue to the use the Git credentials you added above Credentials
QA_TESTS The file containing the Jenkins jobs (e.g. tests.json [1]) String

[1] A sample tests.json file is provided in the qa-jenkins-seed repo.

When you run this job, you should see a new job created for each entry and job trigger in the .json file.

Before doing that, please set the remaining configuration on the job:

Source Code Management

  • Select Git and set the Repository URL to your cloned repo. Set the Credentials to use your Git credentials. To just build the master branch, set the Branch Specifier field to */master.

Build Triggers

  • Select Build when a change is pushed to GitHub.

Build

Complete the Process Job DSLs section:

  1. Set to Look on Filesystem and set the DSL Scripts field to blueprintQA.groovy
  2. Set the Action for removed jobs to delete
  3. Set the Action for removed views to delete

Adding Blueprint Tests

To add an additional blueprint test (or to modify an existing test, such as changing the target locations):

  • Modify the tests.json file to add the details of your new test (and/or update existing tests)

  • Run the Jenkins seed job (e.g. invoke “Build with Parameters”, using the defaults); alternatively wait 30 minutes for this to happen automatically

  • Check that your new build job has been added to the Jenkins job list

  • Kick-off your job, to check that it works

Troubleshooting

    
#!/usr/bin/dumb-init /bin/bash