Phases of Playtesting (Part II)

In Part I of this article, game design researcher Joe Mauriello describes the phases of playtesting as a way to hone the game system.  In Part II, he elaborates on the final stages of playtesting and how to learn from player feedback.

Early Development Test

Develop should progress with testing in mind. That means setting development goals in accordance with the playtesting questions you’ve uncovered from previous sprint playtests. Early in development, the designer should be focused on heart of the game’s system rather then trying to touch on every aspect of the game. An inexperienced game designer might be eager to get their ideas into code right away and see it come to life on the screen. Resist this impulse. Identify your most risky assumption and figure out a way to test it. For example: Perhaps you wish to build an open world RPG with an innovative battle mechanic. Rather than focus on what you know, for example on how the character moves around the open world, figure out the fastest way you can test that battle mechanic. You may just need some paper and a spreadsheet.

In the early phases of testing, it’s important to get to know your system’s dynamics. Dynamics are the result of your designed game mechanics interacting with one another.  In other words, it's your game system brought to life. Observe the players' decisions within the system. Are they in line with your expectations? Is your system breaking down because the player is doing unexpected things? If you are trying to capture a certain aesthetic, or the mood or feeling that arises from the dynamics of the system, but you don’t have much in the way of art, it might be helpful to give them a little flavor and context as to the role they are playing and the world the game is set in. It’s fine to do that, just avoid talking about the system’s dynamics. At this point, these will only be your assumptions. It’s more valuable to watch the dynamics arise through game play.

Test results during this phase will inform game mechanics and get you closer to your design goals. As early development progresses, you’ll be testing less on paper and more on the final system. Even if you have a substantial portion of your game implemented it may still be useful to conduct paper tests every now and again. Especially if something isn’t working and you need to pivot some aspect of the game.

Late Development test

As development progresses and decisions about the game system are being finalized, your focus will shift away from dynamics and towards the player’s comprehension of the game and it’s usability. At this point, playtests become more and more hands off and observation focused. Specifically on sources of confusion, unintended frustration, and the player’s overall feelings towards the game.

The game is further along in development at this point and you may be more heavily invested in it. As the designer, you’ll likely hope that they are enjoying themselves but don’t forget to be objective. Don’t be so eager to finish it. Pay attention to how players are feeling and reacting emotionally. Facial expressions and vocal utterances can tell a lot about how users are experiencing your game. If players aren’t enjoying themselves, be glad you caught it during testing. Playtest Participants can be brutal and silly. They can make suggestions that seem to have nothing to do with the game’s direction and fixate on aspects that you don't feel are important. It's crucial that you leave your preconceived notions and your ego out of the test. Don't try to defend the game and be an objective listener, it’s tougher than it sounds. Users are giving feedback based on their experience so patience is essential here. Focus on how their comments relate to your game. Moments of unintended frustration are issues that need to be addressed.

Test results during this phase should help you clarify the game’s interactions. Perhaps you need to add a bit more polish to the tutorial, maybe some of the interactions in the game require more visual feedback or larger rewards, balancing might need tweaking, etc. Tie up your loose ends before your game is released into the wild.

Making Feedback Useful

Playtesting is a vital part of the iterative process. Write down your findings and consider giving out a survey with scorable questions. Once you've completed a playtest, document the results in order to determine the best course of action. Categorize the feedback collected, identify the area that a player is commenting on and organize it into the different aspects of your game like art, sound, animation, gameplay, feedback, plot, theme, and whatever else might suite your game. Be sure to take note of everything said. Even if something seems ridiculous, it might help give an issue or game play a new perspective. You probably had an idea of what you planned to do next but once you compare that with your feedback you might decide to shift course. Try to have playtests build on one another and compare results from one playtest to the next.

 Joseph Mauriello is an award winning game designer, educator, and developer who's been working in the games industry since 2006. He's created games for Google as well as major motion pictures. Joe currently is helping to usher in the next generation of educational games as a game design researcher for Amplify Learning.

Phases of Playtesting (Part I)

In this article, game design researcher Joe Mauriello describes the phases of playtesting as a way to hone the game system.

Playtesting is a core component of my everyday work as a game design researcher at Amplify Learning.  We are attempting to build the next generation of educational games.  Our mission is to build games that are first and foremost fun, while also conveying learning content and critical thinking through the game system.  While a lot of work has been done in this area, there is no canon of educational games that do this successfully. This leads us to make a lot of assumptions while designing our games.  To help ensure success, we’ve integrated playtesting directly into our development process. We test every two weeks as a sort of capstone to our development sprints. This is the first time I’ve experienced such tight integration of playtesting and development and after seeing the results, I’ve become an advocate.  In the other places I’ve worked, testing has not been so tightly integrated. It wasn’t because people didn’t see value in the process, rather, there was a lack of understanding of how it fit into the development process.   I hope to help clarify that in this article.

Just to clarify, playtesting is not the same as QA testing. In fact, when conducting a test, I generally ignore the bugs unless it interfered in some critical way with what was being tested. The goal of playtesting is to hone the game system.

I find it helpful to break development into three phases each with it’s own testing goals.  It not always obvious when you are shifting from one phase to another. In fact different aspects of the game might be in different phases at the same time.  What’s important here is how to focus your attention and how your findings are valuable to the game’s development:  

1. During pre-production we test our assumptions about concepts and subject matter.  This occurs even before we start the design process. We refer to tests during this phase as mental model tests.  During this phase of development we are selecting and building tools.
2. Early development tests help us understand the dynamics our game system and help us find the fun within them. During this phase of development we are implementing the game system.
3. Late development tests help us find sources of confusion and unintended frustration.  During this phase we tighten the screws and polish the game.

Mental Model Test

A mental model test addresses initial assumptions:  Does our target audience like “X” style of game?  What affordances do they bring to games of this type?  Since we are making educational games, we’ll also test what knowledge our target audience is bringing to the game.  This type of test is akin to market research. 

Don’t be put off from doing this type of test because you have what you think is an original idea. Talk to your peers about your idea.  This type of research can be done through conversation.  Focus on what the player expects from the genre or theme you are working in. Understanding a player’s assumptions will allow you to make informed decisions within the framework of your original idea.  Asking these questions is going to get you thinking about your idea in new dimensions.

Test results during this phase will help you narrow down your concept and find where you want to express yourself through the game’s design.

Joseph Mauriello is an award winning game designer, educator, and developer who's been working in the games industry since 2006. He's created games for Google as well as major motion pictures. Joe currently is helping to usher in the next generation of educational games as a game design researcher for Amplify Learning

March 2013: Pacing

Hello!  I was at a local IGDA meeting when the speaker remarked that great games are not always original ideas, but that the game designers had all the elements, right pacing, and balance to create a great game.  In fact, we have a long honored tradition of building on top of other games, in particular board games and other social games.

I remember loving a casual game called Fairies.  Yes, it was the same gameplay mechanic as Chuzzle. There was just something about the theme, the story, music, and pacing of Fairies that really appealed to me.  While I knew about Chuzzle, the whole fuzzy balls and chemistry beakers never appealed to me and I felt that the difficulty ramped up unevenly.  I have played other classic puzzle games like Puzzle Bobble and have experienced uneven levels.

In bigger games, one might think about pacing as lulls and highs, or variety in gameplay.

• How would you define pacing in videogames?
• How does pacing affect the player experience?
• How do you control pacing in videogames?
• What would you consider to be great pacing?
• What games are good examples of great pacing?

Are Badges Really All That?

In this article, game designer Sande Chen discusses the importance of badges in games, gamification, and in real life.

While working as a game designer, I've had to come up with a lot of badges, titles, or achievements.  A badge or achievement can be a delightful surprise, especially for players who don't go out of their way to collect every badge.  And badges are a great addition for completists, who now get to go through the game again but with an entirely different aim.

Badges are cool if they make the player look at the game another way and encourage exploration.  Badges can track progress and indicate to others how many times a player did an awesome video game feat.  It's all part of our instant infographic world.  But badges are not so cool if they're for lots of repetitive actions (like clicking a button) or paying money into the game. Well, that just shows how much time you've wasted or how much money you've spent, possibly...

Gamification gurus extol the use of badges and sure, people will do things for bragging points.  While status is important, I'm not utterly convinced that badges would motivate someone to do something s/he doesn't want to do.  For instance, if a hotel placed me in a room the furthest away from the elevator and told me I would get the badge of "Hall Strider" for walking down a particular hallway 10 times, I don't think I would care too much about that badge...  unless of course it came with an economic incentive, like 10% off my hotel bill.

Most of the time, in real life, badges or titles are tied to economic or social motivators.  Getting "Employee of the Month" can translate into better performance reviews and a raise.  And with each move up the corporate ladder, you get a new, distinguished title.  If you get other badges or titles at work unrelated to economic incentive, it's probably all in fun, some kind of in-joke among co-workers.

If you're aiming to be Mayor of an establishment on Foursquare, it could be for the discounts the place gives to the Mayor,  like any business would give to a frequent customer.  Or it could be a social rivalry among friends.  If suppose a celebrity went on a talk show and declared it cool to get "Hall Strider," I bet the hotel wouldn't have any trouble getting people to accept that room now that the badge of "Hall Strider" has social status.  No doubt people would aim for "Hall Stalker" and further.

Yes, designers can successfully incorporate these social rivalries and instead of economic motivators, give gameplay motivators.  Badges can be fun elements, but badges in itself do not equal fun.  Just because there are badges doesn't mean people will want those badges.  If your gamification effort is based upon giving out badges, then take a look at the core activity and ask yourself, is this fun?  If you're giving out badges for something completely not fun like undergoing root canals, the simple act of giving out badges or smiley stickers is not going to drive up demand for root canals.  That's when it might be a good time to re-evaluate the project and see how gamification, not badges, can help you.

Sande Chen is a writer and game designer whose work has spanned 10 years in the industry. Her credits include 1999 IGF winner Terminus, 2007 PC RPG of the Year The Witcher, and Wizard 101. She is one of the founding members of the IGDA Game Design SIG.