From pre production testing to post production monitoring

When a new product or feature is being developed, it is critical to have automated tests running regularly to catch regression as changes are committed and retested. Using Tesults (tesults.com) for test results reporting helps your team during prerelease testing and also sets up a process for production monitoring too. By the time a new product or feature is ready to ship the work needed for production monitoring reporting and notifications has already been done.

Pre production testing

At this stage, product development is underway and test authoring is ideally being carried out alongside it. You may be using a popular test framework such as JUnit, Pytest, Robot, Mocha, Jest etc. or you might be using a custom test framework. Either way, integrating Tesults takes little time. Libraries are provided by Tesults to make this process straightforward.

The most important planning decision during this time is to ensure your test jobs map appropriately to Tesults reporting targets. A Tesults target typically map 1:1 with a test job and how many targets you create is totally up to you and what makes sense for your project. You may choose to create a single target for each client you support, for example one for the web client, one for the iOS client and one for the Android client. Or you may choose to be more specific and have targets per client and per product feature or area, for example, you may have a target focused entirely on authentication or sign up onboarding. Alternatively you may just report everything into one target and group tests by test suite instead. Whatever you decide when you review your results they should make sense to you and your team.

Lead up to production release

In the lead up to a production release when your team is testing and bug fixing, test results output needs to be carefully monitored. The supplemental view on Tesults makes this easier by automatically analyzing output for new failures, new fixes and continuing failures.

At this stage you may want to start sending results from Tesults to Slack, Mattermost or MS Teams or simply enable email notifications to keep stakeholders in a tight feedback loop. This is easily done via the configuration menu. It’s also time to start considering whether the same tests you are running in prerelease need to be run in production too. In some cases tests need to be modified for production to be focused on testing uptime and availability rather than be focused on comprehensive feature functionality testing. Now is is a good time to ensure production tests are authored and what is great is that you can use the existing test infrastructure / framework you have already setup to do this and simply create new targets on Tesults for production.

Release and post production monitoring

Following an exciting and successful product release (well done!) your team’s attention can turn to production monitoring. At this point however, there is very little left to do if you have been using Tesults for prerelease test output. You probably want to ensure notifications (email, Slack, MS Teams, Mattermost) are enabled and perhaps add PagerDuty alerting from the configuration menu to text or call on-call developers but the fundamentals of monitoring are already in place. Another benefit from having Tesults integrated is that you can utilize the Tesults API to fetch test results data for decision making within your continuous integration and deployment pipeline.

Use Tesults for pre production regression test reporting and analysis and get post production monitoring and notifications immediately with little to no extra effort.

- Tesults Team