Phases of Playtesting (Part II)

In Part I of this article, game design researcher Joe Mauriello describes the phases of playtesting as a way to hone the game system.  In Part II, he elaborates on the final stages of playtesting and how to learn from player feedback.

Early Development Test

Develop should progress with testing in mind. That means setting development goals in accordance with the playtesting questions you’ve uncovered from previous sprint playtests. Early in development, the designer should be focused on heart of the game’s system rather then trying to touch on every aspect of the game. An inexperienced game designer might be eager to get their ideas into code right away and see it come to life on the screen. Resist this impulse. Identify your most risky assumption and figure out a way to test it. For example: Perhaps you wish to build an open world RPG with an innovative battle mechanic. Rather than focus on what you know, for example on how the character moves around the open world, figure out the fastest way you can test that battle mechanic. You may just need some paper and a spreadsheet.

In the early phases of testing, it’s important to get to know your system’s dynamics. Dynamics are the result of your designed game mechanics interacting with one another.  In other words, it's your game system brought to life. Observe the players' decisions within the system. Are they in line with your expectations? Is your system breaking down because the player is doing unexpected things? If you are trying to capture a certain aesthetic, or the mood or feeling that arises from the dynamics of the system, but you don’t have much in the way of art, it might be helpful to give them a little flavor and context as to the role they are playing and the world the game is set in. It’s fine to do that, just avoid talking about the system’s dynamics. At this point, these will only be your assumptions. It’s more valuable to watch the dynamics arise through game play.

Test results during this phase will inform game mechanics and get you closer to your design goals. As early development progresses, you’ll be testing less on paper and more on the final system. Even if you have a substantial portion of your game implemented it may still be useful to conduct paper tests every now and again. Especially if something isn’t working and you need to pivot some aspect of the game.

Late Development test

As development progresses and decisions about the game system are being finalized, your focus will shift away from dynamics and towards the player’s comprehension of the game and it’s usability. At this point, playtests become more and more hands off and observation focused. Specifically on sources of confusion, unintended frustration, and the player’s overall feelings towards the game.

The game is further along in development at this point and you may be more heavily invested in it. As the designer, you’ll likely hope that they are enjoying themselves but don’t forget to be objective. Don’t be so eager to finish it. Pay attention to how players are feeling and reacting emotionally. Facial expressions and vocal utterances can tell a lot about how users are experiencing your game. If players aren’t enjoying themselves, be glad you caught it during testing. Playtest Participants can be brutal and silly. They can make suggestions that seem to have nothing to do with the game’s direction and fixate on aspects that you don't feel are important. It's crucial that you leave your preconceived notions and your ego out of the test. Don't try to defend the game and be an objective listener, it’s tougher than it sounds. Users are giving feedback based on their experience so patience is essential here. Focus on how their comments relate to your game. Moments of unintended frustration are issues that need to be addressed.

Test results during this phase should help you clarify the game’s interactions. Perhaps you need to add a bit more polish to the tutorial, maybe some of the interactions in the game require more visual feedback or larger rewards, balancing might need tweaking, etc. Tie up your loose ends before your game is released into the wild.

Making Feedback Useful

Playtesting is a vital part of the iterative process. Write down your findings and consider giving out a survey with scorable questions. Once you've completed a playtest, document the results in order to determine the best course of action. Categorize the feedback collected, identify the area that a player is commenting on and organize it into the different aspects of your game like art, sound, animation, gameplay, feedback, plot, theme, and whatever else might suite your game. Be sure to take note of everything said. Even if something seems ridiculous, it might help give an issue or game play a new perspective. You probably had an idea of what you planned to do next but once you compare that with your feedback you might decide to shift course. Try to have playtests build on one another and compare results from one playtest to the next.

 Joseph Mauriello is an award winning game designer, educator, and developer who's been working in the games industry since 2006. He's created games for Google as well as major motion pictures. Joe currently is helping to usher in the next generation of educational games as a game design researcher for Amplify Learning.

Phases of Playtesting (Part I)

In this article, game design researcher Joe Mauriello describes the phases of playtesting as a way to hone the game system.

Playtesting is a core component of my everyday work as a game design researcher at Amplify Learning.  We are attempting to build the next generation of educational games.  Our mission is to build games that are first and foremost fun, while also conveying learning content and critical thinking through the game system.  While a lot of work has been done in this area, there is no canon of educational games that do this successfully. This leads us to make a lot of assumptions while designing our games.  To help ensure success, we’ve integrated playtesting directly into our development process. We test every two weeks as a sort of capstone to our development sprints. This is the first time I’ve experienced such tight integration of playtesting and development and after seeing the results, I’ve become an advocate.  In the other places I’ve worked, testing has not been so tightly integrated. It wasn’t because people didn’t see value in the process, rather, there was a lack of understanding of how it fit into the development process.   I hope to help clarify that in this article.

Just to clarify, playtesting is not the same as QA testing. In fact, when conducting a test, I generally ignore the bugs unless it interfered in some critical way with what was being tested. The goal of playtesting is to hone the game system.

I find it helpful to break development into three phases each with it’s own testing goals.  It not always obvious when you are shifting from one phase to another. In fact different aspects of the game might be in different phases at the same time.  What’s important here is how to focus your attention and how your findings are valuable to the game’s development:  

1. During pre-production we test our assumptions about concepts and subject matter.  This occurs even before we start the design process. We refer to tests during this phase as mental model tests.  During this phase of development we are selecting and building tools.
2. Early development tests help us understand the dynamics our game system and help us find the fun within them. During this phase of development we are implementing the game system.
3. Late development tests help us find sources of confusion and unintended frustration.  During this phase we tighten the screws and polish the game.

Mental Model Test

A mental model test addresses initial assumptions:  Does our target audience like “X” style of game?  What affordances do they bring to games of this type?  Since we are making educational games, we’ll also test what knowledge our target audience is bringing to the game.  This type of test is akin to market research. 

Don’t be put off from doing this type of test because you have what you think is an original idea. Talk to your peers about your idea.  This type of research can be done through conversation.  Focus on what the player expects from the genre or theme you are working in. Understanding a player’s assumptions will allow you to make informed decisions within the framework of your original idea.  Asking these questions is going to get you thinking about your idea in new dimensions.

Test results during this phase will help you narrow down your concept and find where you want to express yourself through the game’s design.

Joseph Mauriello is an award winning game designer, educator, and developer who's been working in the games industry since 2006. He's created games for Google as well as major motion pictures. Joe currently is helping to usher in the next generation of educational games as a game design researcher for Amplify Learning

February 2013: Playtesting

February 2013's topic was submitted by game designer Pascal Belanger.


Play testing, (may it be iterative or planned with a group of your target audience) is a process more and more used by companies.

My very first contact with the concept was when I heard that Valve was play testing their games as soon as the first prototype was made.  This is part of their design philosophy and we can learn a lot from them on that part.  For instance, it has helped them to make the flow of Portal's learning curve as seamless as possible.

• What importance do you attribute to play testing? 
• What part does it take in your design flow? 
•  How often do you do it?

An article about the relationship between Portal and play testing : http://www.develop-online.net/news/29500/Playtesting-was-crucial-to-Portals-success