Hi!
I’m curious about your experience or opinion on test automation in game development.
My questions are:
If you have any thoughts or real stories about it, I’d love to hear them.
What do you mean by “what is your profit”? Can you elaborate?
I’ve not been involved in game testing / automation. But I did come across this before: https://www.gamedriver.io. That’s all I’m aware of for publicly disclosed game test automation tools.
Either other tools are niche tools that require crafty internet searching to find, or are home grown and not publicly disclosed.
Thank you for the link.
There are some good frameworks for automation in game development, for example, Alt Unity.
In general, it is an addition to the game engine.
I’m referring to ROI when I talk about profit. I saw when automation was used for performance and backend testing. It was useful. It is often useless for UI and regression testing. The benefit is not justified.
I would like to know if there is a positive case.
I once interviewed for a game QA role, and all I could work out, is that everything is bespoke, nothing comes off of the shelf, it’s far worse in gamedev automation than it is in mobile automation. That’s not to say that either industry is low on quality due to lack of tool standardization or tool commodification and formalizing. But rather that, yeah it is going to be hard. Do search about here, a fair few of the MOT contributors do work in games QA or have games experience.
Are you able to elaborate on your observations/findings, in terms of why the benefit was not justified for ROI?
I may stand corrected but I would deduce the reasoning this is so is due to the complexity of automating against a game-based UI (in modern times often 3d visual environment, game specific controls, camera angle panning & tilting) compared to the simpler tasks of automating more simplistic actions to test performance and backend of the game, where these latter tasks are often API or code/library invocation based, bypassing or only minimally involving the UI. The tradeoff in effort & complexity makes it not worthwhile for ROI.
Game UIs, unless using common frameworks/engines, are custom and that makes it hard to automate for. And even when using common frameworks (e.g. Unity, Unreal engine), unless the community (the sponsoring company of the framework, or savvy altruistic rockstar open source developers) contributes automation tooling for it, you won’t really get anything, much less anything for free since the effort involved in this usually doesn’t justify open sourcing such tooling when available.
It’s nice that there is Alt Unity that is open source. Thanks for the share of that info. If only more game dev communities/companies followed that approach, maybe we would get more tooling available and as such that could assist in making ROI benefits easier to achieve.
I just came across this post that goes over a list of tools that covers #1 of your question.