Result interpretation
When using a test framework integration to report results Tesults automatically maps test status values from a test framework to one of three Tesults result status values: pass, fail and unknown. Typically all passing results are mapped to pass, all failing results are mapped to fail, and everything else is mapped to unknown, including states such as skipped, timed out and any other test framework specific status values that are not clearly passing or failing states.
To have finer control over this mapping and expose test framework specific result values in the interface, use the Result Interpretation feature.
Steps to using result interpretation
- Most test framework integrations do not currently support result interpretation. Over time more integrations will add support. Currently support is limited to these integrations:IntegrationMinimum versionSuggested minimum mappingPytest1.4.0+passed→passfailed→failMocha1.2.0+passed→passfailed→failJasmine1.0.1+passed→passfailed→failPlaywright1.0.1+passed→passfailed→failCodeceptJS1.1.0+passed→passsuccess→passfailed→failNightwatch1.1.0+passed→passfailed→failWaffle (using mocha-tesults-reporter)1.2.0+passed→passfailed→failCypress1.3.0+passed→passfailed→fail
If you are using a language library (Java, Python, JavaScript etc.), or are the author of a custom test framework or you are an integration developer use the rawResult property for test cases to enable support for result interpretation. - Go to the configuration menu and select result interpretation. Click the 'Create new results mapping' button to create a new mapping.
- Creating a mapping of raw results values (from the test framework) to either a pass or fail result in Tesults. Anything not mapped will be mapped to unknown.
- Go back to the configuration menu and select targets. Select the target you wish to apply the result interpretation mapping to. Look for the 'Target result interpretation map' field. By default this will be set to None. Select Edit and select the map you created in the previous step.
- Done. View results now and you will see that results are being interpreted in the way you have defined in your map. You can always edit the mapping you have created if you want to adjust what status values you treat as pass or fail.
