Test reporting and continuous integration and deployment

When triggering tests in a continuous integration and deployment (CI/CD) pipeline there may be times you want to make a deployment decision based on test results. You may also want to share build and test failures with your team. This post explains how to do this with Tesults.

The assumption for this post is that you have already signed up, created a project and integrated your tests to report results with Tesults. If you have not please take look at getting started. You should also have your continuous integration system set up to trigger your tests at the appropriate time in your build pipeline.

To make a deployment decision based on test results, or to share build and test failures, your continuous integration system needs to be able to access the relevant test results for the current build. This is done using the Tesults API.

To make it easier to retrieve results data it is recommended you pass in a build name and build result as part of your Tesults integration. This build name should also be available to your continuous integration pipeline. Instructions on how to pass in build information to Tesults are dependent on the test framework integration or language library you are using, please review the documentation for your language or test framework. It is possible to retrieve results without using a build name but this is less straight forward because it requires checking the timestamp for the test run to determine which results you are interested in.

Create a Tesults API token

Go to the configuration menu and select API Token from the menu.

Click the Create API Token button to create an API token.

Copy the token for use with API requests.

Retrieve results data

Utilize the /api/results endpoint (reference documentation) to retrieve results data after your test run step in the continuous integration pipeline has completed. To do this make a GET request to https://www.tesults.com/api/results. Pass your API token as an authorization header and specify the target id for which to fetch results.

A curl request to fetch data looks like this:

curl https://www.tesults.com/api/results
?target=target_id&build=build_name;
-H "Authorization: Bearer token"

Replace token with the API token
Replace target_id with a target ID
Replace build_name with a build name

The request above assumes you are using a build name (any string value) to retrieve results. If you are not using a build name, omit the build parameter to instead fetch the latest results data for the target. You can utilize the limit parameter to fetch more than one set of results. Please view the Results API documentation for a full list of parameters. View Targets API documentation to learn how to retrieve target IDs.

A successful response will retrieve results data:

{
    "data": {
        "code": 200,
        "message": "Success",
        "results":{
            "runs": [
                {
                    "created_time": 1631398688540,
                    "pass": 149,
                    "total": 150,
                    "build_name": "1.0.0",
                    "build_result": "pass",
                    "build_raw_result": "passed",
                    "cases": [
                        {
                            "name": "Sign in successfully",
                            "desc": "Signs in to site",
                            "suite": "Authentication",
                            "result": "pass",
                            "duration": 11122000,
                            "num": 0,
                            "hash": "7c95f9b7-578d92c19fdd",
                            "_CustomField": "Custom value",
                            "params": {
                                "terrain": "land",
                                "climate": "warm",
                                "temperature": "25C"
                             },
                            "files": [
                                "screenshot1.png", "test.log"
                            ],
                            "files_base_path": "/files/base/path"
                        },
                        {... more test cases}
                    ]
                ]
            },
           "cursor": cursor_value
       }
}

Parse the JSON and retrieve the run object. Use the pass and total fields to determine if all test cases passed and if you want more detailed information on specific failures retrieve the cases array from the response object.

What you ultimately base a deployment decision on is up to you and your team. This may be as simple as ensuring all tests have passed (pass == total) or you may have additional checks such as ensuring that the total number of test cases is larger than zero. Whatever the criteria you use you now have the data you need to make an informed deployment decision and provide success and failure feedback.

Please contact help@tesults.com if you have questions or require assistance.

- Tesults Team