Business Model Changes in the Video Game Industry Lead to a Better ROI for Test Automation

Test automation requires considerable up front investment in terms of time and resource. A test harness or automated test framework has to be written and integrated. Continuous integration with build scripts requires configuration. Developers must write product code that is modular and testable and provide the ability to add test hooks. Then the tests themselves must be written. Finally test results must be collated and reported, which is what we here at Tesults take care of doing.

For most software projects the returns on investment are far higher than the initial cost which is why test automation has become so commonplace across software companies over the years. The powerful regression testing ability that automation provides build after build would require enormous manual resource to do and in most cases it would be simply too cost prohibitive and elongate release timelines.

The video game industry has lagged behind other types of software companies in embracing test automation. Any developer or QA manager that has worked both inside and out of the games industry would have likely noticed this discrepancy. The main reason for this is historical. Even a decade ago it was common to release a new game title and then largely abandon the code base and move onto another project. There was never enough time over the development life cycle to recuperate the initial cost of test automation. In other industries such as productivity software, even back as late as the 1980s and 90s subsequent releases and updates were iterative. This meant regression testing was paying back the investment every single day for years and even decades producing huge returns.

Looking at the games industry now, there has been a change in business model to a more service based approach especially on mobile. Traditional console style game companies increasingly release downloadable content. Even those that continue to release boxed titles every year often maintain their core engine code base for longer periods of time, for some franchises this can be up to a decade. For these reasons test automation is increasingly becoming something game studios are looking to implement. The increasingly complexity of games is making it necessary to complement human testers with automated solutions for both labour cost and time reasons. There are also certain types of tests such as memory soak, network and server load testing required for social games that cannot be easily done through manual testing and there has been a greater adoption of automation.

The games industry is highly competitive. The old business model of games development is in trouble and in decline. There will be a greater service based approach in the future and a greater need for regression testing. The only reason automation lags behind other industries is due to a culture that has been able to persist due to the slow transition to a new business model. The economics now make test automation return on investment so high it makes sense for almost everyone to be doing it if they are not already.

Having been developers and developers in test ourselves, including at game studios, we have noticed that the economic reality of the necessity of test automation has now largely been realized by management and is no longer in dispute. However while adoption is occurring it is still slower and less effective than it should be due to a lack of experience in the industry about how to implement it along with a persistence of a reluctant culture among participants inside and outside of programming teams. At Tesults we have found the greatest driver for increasing test coverage and adoption is through the use of highly visible test reporting. Test results should be available continuously, every build for every build flavor and platform. Leads and producers should be checking test results alongside their production schedules throughout the sprint.

The high ROI with test automation in the game industry is no longer in dispute. Do it now if you are not already. If you have begun and want to ensure success, effectiveness and high adoption take a look at Tesults for handling your test results.