In the rapidly evolving world of game development, new technologies that can potentially revolutionize the way we work are constantly emerging. Every month, sometimes even every week, there’s some new tool, feature or framework that can potentially alter the way games are made, released and, of course, playtested.
One such innovation is the introduction of artificial intelligence (AI) powered analysis in the game design and research process–as you can see by our most recent feature, the AI-Powered Analysis. Gareth Lloyd, a seasoned games user researcher, recently shed light on the potential benefits, use-cases, and implications of this tool.
In his mind, AI-Powered Analysis offers an invaluable tool for developers, particularly during playtesting. This tool should not be considered a replacement for human analysis but as an enhancement of it–a tool that will save you both time and effort, while not replacing the need for human analysis. Now that AI-analysis is in the mix, one question arises: how can a games user researcher use it?
To find out, we sat down with Gareth to hear all about how he's using our new tool. Gareth's typical approach involves starting with a single playtest video, reviewing and annotating it manually before switching on the AI-Powered Analysis tool to assess additional videos. "I will still watch one video and annotate it with the AI turned off because for me, that first look without any input is too valuable." Gareth remarked. Yet, with AI, "I'm able to go through the other videos at a higher speed." The AI-Powered Analysis tool in this scenario helps highlight interesting moments and aligns these with Gareth's prior annotations.
Notably, the AI-Powered Analysis tool also allows developers to go through the videos at a faster pace, making it an effective tool for efficiency in the reviewing process. This might not sound monumental initially, but the implications are enormous. Moving from an entirely empty playtest–a daunting prospect at first glance–to one already filled with preliminary notes and insights on player behavior and potential issues is a game-changer. This way, AI assists the researcher in identifying patterns and issues early, and it serves as a way to make what can often be a long and complex process easier and more efficient. The tool is useful not only in identifying key moments in gameplay but also in summarizing the player experience, spotting potential issues, and generally accelerating the pace of playtest analysis.
However, in the balance between AI and human analysis, AI-Powered Analysis is primarily an asset in your toolbox. "AI analysis will never replace human analysis, but it will help humans to do research a bit faster", it's not meant to undermine or replace the important work of a human researcher's analysis. It comes into its own by reducing the hours spent on the initial part of analyzing many playtest videos, offering a headstart in understanding player behavior, spotting issues, and indicating areas of focus.
That being said, a major point of contention in using AI-Powered Analysis–or AI tools in general–is the potential risk of bias and the need to maintain scientific integrity. Gareth's perspective on this is that it's crucial to incorporate processes of review and verification to counterbalance the influence of AI. "I've been calling my processes with it with the AI triaging cause it helps me locate interesting bits in the video and even sometimes it gives me a steer on what is actually happening in those moments."
Nevertheless, it's crucial to remember that the AI-Powered Analysis tool is not the source of insight. Instead, it’s the user who extracts insights from these annotations, ensuring the maintenance of integrity and avoidance of bias.
In general, according to Lloyd, AI-Powered Analysis can be valuable to both newcomers and industry veterans. The justified skepticism that many, including Gareth himself, might initially bring to such a new technology can be constructively channeled into critical evaluation of the tool and its effectiveness. He envisions the tool as a means of integrating cutting-edge technology into traditional workflows and facilitating efficiency and effectiveness–keeping the industry on the edge of technology.
"I came in from very much like a skeptic position as I think everyone should with this. Like maybe it's a me thing, but I think it's the way that you should look at this. But we've tried to build something that, that supports you doing games, user research, it helps you avoid like a bit of the, the, the most time consuming part of analyzing a play test highlights potential issues. We think it does quite a good job with it. It's not a replacement for watching your videos with a researcher's eye here, but it's a pretty effective tool for for, for, for doing that faster and ensuring you don't miss Key moments"
All in all, AI-Powered Analysis can significantly augment the process of game user research, but its effectiveness ultimately hinges on how you as researchers decide to utilize it. Its main value lies in accelerating the pace of review, ensuring key moments aren’t missed, and offering early insights, rather than completely taking over the process. Despite the great strides AI has made recently, the keen eye of a human researcher remains irreplaceable, and Gareth's insights highlight the importance of finding the right balance between human analysis and AI support. Don't just take our word for it though: try it today. Equip your game development arsenal by enabling AI-Powered Analysis and blaze through your playtest reviews, as your games level up at breakneck speed.
