Documentation
Pricing
Sign in
Sign up
 

Github Actions

Running tests and reporting results within a continuous integration (CI) is a common task. If you use Github Actions and your workflow includes a job with a step that runs your tests, you may need to check the result to determine whether the pipeline is a success or a failure.

Some test frameworks automatically fail the process if the tests do not all pass. Others do not. For the latter case Tesults provides the Tesults API results check Action in the Github Actions Marketplace to check results and usage is demonstrated below. https://github.com/marketplace/actions/tesults-api-results-check

In the case of the Playwright test framework for example the step will report results to Tesults and then Playwright will automatically fail the job if all test cases do not pass. In this case you do not need to do anything else.

name: Playwright Tests
on:
push:
  branches: [ main, master ]
pull_request:
  branches: [ main, master ]
jobs:
test:
  timeout-minutes: 60
  runs-on: ubuntu-latest
  steps:
  - uses: actions/checkout@v3
  - uses: actions/setup-node@v3
  with:
    node-version: 18
  - name: Install dependencies
  run: npm ci
  - name: Install Playwright Browsers
  run: npx playwright install --with-deps
  - name: Run Playwright tests
    env:
      TARGET: ${{ secrets.TARGET }}
      BUILD: ${{ format('{0} {1}', 'Build ', github.run_number) }}
    run: npx playwright test

In the case above this assumes the playwright.config.js file contains reporter parameters like this:

reporter: [['playwright-tesults-reporter',
{'tesults-target': process.env.TARGET,
'tesults-build-name': process.env.BUILD,
'tesults-build-result': 'pass'}]],

This all works well for the Playwright test framework because it handles returning the appropriate process code for failing the job.

In the case where the test framework does not automatically fail the job, after tests have run and results data has been uploaded to Tesults, you would want to check results via the Tesults API, and then determine whether to fail the job or not.

To make this easy for Github Actions Tesults has made the Tesults API results check Action available in the Github Actions Marketplace:https://github.com/marketplace/actions/tesults-api-results-check

Using this action is demonstrated below.

- name: Check
id: check
uses: tesults/tesults-api-results-check@1.0.6
with:
  target: ${{ secrets.TARGET }}
  build: ${{ format('{0} {1}', 'Build ', github.run_number) }}
  api_token: ${{ secrets.API_TOKEN }}

The target, build and api_tokens are all required.

  1. target - the id of the target (available from the Tesults configuration menu (https://www.tesults.com/config) and Tesults API)
  2. build - the build name, this is self generated and should be unique for the test run, we recommend simply using github.run_number
  3. api_token - the api token for the Tesults project, generated from the Tesults configuration menu (https://www.tesults.com/config)

This action will automatically fail the job if the pass rate is not 100%, however if you need the number of passes and total for any reason, the action outputs these values and they can be examined.

# Use the output from the `Check` step
- name: Get the number of passes and total
run: echo "${{ steps.check.outputs.pass }} pass out of ${{ steps.check.outputs.total }} total"

View the output in the Actions tab on Github.

github-actions-output