April 2011 Poll

Please vote for the April 2011 topic! As always, feel free to suggest more topics!  Look at the submission guidelines for Topics and Blog Entries.

You'll see the poll to the right. The choices are:

• Gamification
• Emergence
• Dealing with Communities (How Fans Affect Our Games)

Emergence (from Pascal Bélanger)
What is emergence to you? How do you deal with it? What place do you let it take in your games/design? And in your opinion, can we make emergent games that would still be art? Or even better, are there any valid existing examples of such games.
Jesper Juul on Emergence vs progression: http://www.jesperjuul.net/text/openandtheclosed.html
 
Please vote by March 28.  Thank you!

Metrics (Part II)

In Part I, game designer Ian Schreiber outlines the debate between metrics-driven design and the more touchy-feely intuition-based design. In Part II, he explains the difficulties with trying to measure the "fun" in your game.

How much to measure?

Suppose you want to take some metrics in your game so you can go back and do statistical analysis to improve your game balance. What metrics do you actually take – that is, what exactly do you measure?

There are two schools of thought that I’ve seen. One is to record anything and everything you can think of, log it all, mine it later. The idea is that you’d rather collect too much information and not use it, than to not collect a piece of critical info and then have to re-do all your tests.

Another school of thought is that “record everything” is fine in theory, but in practice you either have this overwhelming amount of extraneous information from which you’re supposed to find this needle in a haystack of something useful, or potentially worse, you mine the heck out of this data mountain to the point where you’re finding all kinds of correlations and relationships that don’t actually exist. By this way of thinking, instead you should figure out ahead of time what you’re going to need for your next playtest, measure that and only that, and that way you don’t get confused when you look at the wrong stuff in the wrong way later on.

Again, think about where you stand on the issue.

Personally, I think a lot depends on what resources you have. If it’s you and a few friends making a small commercial game in Flash, you probably don’t have time to do much in the way of intensive data mining, so you’re better off just figuring out the useful information you need ahead of time, and add more metrics later if a new question occurs to you that requires some data you aren’t tracking yet. If you’re at a large company with an army of actuarial statisticians with nothing better to do than find data correlations all day, then sure, go nuts with data collection and you’ll probably find all kinds of interesting things you’d never have thought of otherwise.

What specific things do you measure?

That’s all fine and good, but whether you say “just get what we need” or “collect everything we can,” neither of those is an actual design. At some point you need to specify what, exactly, you need to measure.

Like game design itself, metrics is a second-order problem. Most of the things that you want to know about your game, you can’t actually measure directly, so instead you have to figure out some kind of thing that you can measure that correlates strongly with what you’re actually trying to learn.

Example: measuring fun

Let’s take an example. In a single-player Flash game, you might want to know if the game is fun or not, but there’s no way to measure fun. What correlates with fun, that you can measure? One thing might be if players continue to play for a long time, or if they spend enough time playing to finish the game and unlock all the achievements, or if they come back to play multiple sessions (especially if they replay even after they’ve “won”), and these are all things you can measure. Now, keep in mind this isn’t a perfect correlation; players might be coming back to your game for some other reason, like if you’ve put in a crop-withering mechanic that punishes them if they don’t return, or something. But at least we can assume that if a player keeps playing, there’s probably at least some reason, and that is useful information. More to the point, if lots of players stop playing your game at a certain point and don’t come back, that tells us that point in the game is probably not enjoyable and may be driving players away. (Or if the point where they stopped playing was the end, maybe they found it incredibly enjoyable but they beat the game and now they’re done, and you didn’t give a reason to continue playing after that. So it all depends on when.)

Player usage patterns are a big deal, because whether people play, how often they play, and how long they play are (hopefully) correlated with how much they like the game. For games that require players to come back on a regular basis (like your typical Facebook game), the two buzzwords you hear a lot are Monthly Active Uniques and Daily Active Uniques (MAU and DAU). The “Active” part of that is important, because it makes sure you don’t overinflate your numbers by counting a bunch of old, dormant accounts belonging to people who stopped playing. The “Unique” part is also important, since one obsessive guy who checks FarmVille ten times a day doesn’t mean he counts as ten users. Now, normally you’d think Monthly and Daily should be equivalent, just multiply Daily by 30 or so to get Monthly, but in reality the two will be different based on how quickly your players burn out (that is, how much overlap there is between different sets of daily users). So if you divide MAU/DAU, that tells you something about how many of your players are new and how many are repeat customers.

For example, suppose you have a really sticky game with a small player base, so you only have 100 players, but those players all log in at least once per day. Here your MAU is going to be 100, and your average DAU is also going to be 100, so your MAU/DAU is 1. Now, suppose instead that you have a game that people play once and never again, but your marketing is good, so you get 100 new players every day but they never come back. Here your average DAU is still going to be 100, but your MAU is around 3000, so your MAU/DAU is about 30 in this case. So that’s the range, MAU/DAU goes between 1 (for a game where every player is extremely loyal) to 28, 30 or 31 depending on the month (representing a game where no one ever plays more than once).

A word of warning: a lot of metrics, like the ones Facebook provides, might use different ways of computing these numbers so that one set of numbers isn’t comparable to another. For example, I saw one website that listed the “worst” MAU/DAU ratio in the top 100 applications as 33-point-something, which should be flatly impossible, so clearly the numbers somewhere are being messed with (maybe they took the Dailies from a different range of dates than the Monthlies or something). And then some people compute this as a %, meaning on average, what percentage of your player pool logs in on a given day, which should range from a minimum of about 3.33% (1/30 of your monthly players logging in each day) to 100% (all of your monthly players log in every single day). This is computed by taking DAU/MAU (instead of MAU/DAU) and multiplying by 100 to get a percentage. So if you see any numbers like this from analytics websites, make sure you’re clear on how they’re computing the numbers so you’re not comparing apples to oranges.

Why is it important to know this number? For one thing, if a lot of your players keep coming back, it probably means you’ve got a good game. For another, it means you’re more likely to make money on the game, because you’ve got the same people stopping by every day… sort of like how if you operate a brick-and-mortar storefront, an individual who just drops in to window-shop may not buy anything, but if that same individual comes in and is “just looking” every single day, they’re probably going to buy something from you eventually.

[This article is an excerpt from Level 8: Metrics and Statistics, part of Ian Schreiber's course on game balance called Game Balance Concepts.]

Ian Schreiber has been making games professionally since 2000, first as a programmer and then as a game designer. He currently teaches game design classes for Savannah College of Art and Design and Columbus State Community College. He has worked on five shipped games and hundreds of shipped students. You can learn more about Ian at his blog, Teaching Game Design.

Metrics (Part I)

In this article, game designer Ian Schreiber outlines the debate between metrics-driven design and the more touchy-feely intuition-based design.

Metrics

Here’s a common pattern in artistic and creative fields, particularly things like archaeology or art preservation or psychology or medicine where it requires a certain amount of intuition but at the same time there is still a “right answer” or “best way” to do things. The progression goes something like this:

1. Practitioners see their field as a “soft science”; they don’t know a whole lot about best principles or practices. They do learn how things work, eventually, but it’s mostly through trial and error.
2. Someone creates a technology that seems to solve a lot of these problems algorithmically. Practitioners rejoice. Finally, we’re a hard science! No more guesswork! Most younger practitioners abandon the “old ways” and embrace “science” as a way to solve all their field’s problems. The old guard, meanwhile, sees it as a threat to how they’ve always done things, and eyes it skeptically.
3. The limitations of the technology become apparent after much use. Practitioners realize that there is still a mysterious, touchy-feely element to what they do, and that while some day the tech might answer everything, that day is a lot farther off than it first appeared. Widespread disillusionment occurs as people no longer want to trust their instincts because theoretically technology can do it better, but people don’t want to trust the current technology because it doesn’t work that great yet. The young turks acknowledge that this wasn’t the panacea they thought; the old guard acknowledge that it’s still a lot more useful than they assumed at first. Everyone kisses and makes up.
4. Eventually, people settle into a pattern where they learn what parts can be done by computer algorithms, and what parts need an actual creative human thinking, and the field becomes stronger as the best parts of each get combined. But learning which parts go best with humans and which parts are best left to computers is a learning process that takes a while.

Currently, game design seems to be just starting Step 2. We’re hearing more and more people anecdotally saying why metrics and statistical analysis saved their company. We hear about MMOs that are able to solve their game balance problems by looking at player patterns, before the players themselves learn enough to exploit them. We hear of Zynga changing the font color from red to pink which generates exponentially more click-throughs from players to try out other games. We have entire companies that have sprung up solely to help game developers capture and analyze their metrics. The industry is falling in love with metrics, and I’ll go on record predicting that at least one company that relies entirely on metrics-driven design will fail, badly, by the time this whole thing shakes out, because they will be looking so hard at the numbers that they’ll forget that there are actually human players out there who are trying to have fun in a way that can’t really be measured directly. Or maybe not. I’ve been wrong before.

At any rate, right now there seems to be three schools of thought on the use of metrics:

• The old school Zynga model: design almost exclusively by metrics. Love it or hate it, 60 Million monthly active unique players laugh at your feeble intuition-based design.
• Rebellion against the old school Zynga model: metrics are easy to misunderstand, easy to manipulate, and are therefore dangerous and do more harm than good. If you measure player activity and find out that more players use the login screen than any other in-game action, that doesn’t mean you should add more login screens to your game out of some preconceived notion that if a player does it, it’s fun. If you design using metrics, you push yourself into designing the kinds of games that can be designed solely by metrics, which pushes you away from a lot of really interesting video game genres.
• The moderate road: metrics have their uses, they help you tune your game to find local “peaks” of joy. They help you take a good game and make it just a little bit better, by helping you explore the nearby design space. However, intuition also has its uses; sometimes you need to take broad leaps in unexplored territory to find the global “peaks,” and metrics alone will not get you there, because sometimes you have to make a game a little worse in one way before it gets a lot better in another, and metrics won’t ever let you do that.

Think about it for a bit and decide where you stand, personally, as a designer. What about the people you work with on a team (if you work with others on a team)?

[This article is an excerpt from Level 8: Metrics and Statistics, part of Ian Schreiber's course on game balance called Game Balance Concepts.]

Ian Schreiber has been making games professionally since 2000, first as a programmer and then as a game designer. He currently teaches game design classes for Savannah College of Art and Design and Columbus State Community College. He has worked on five shipped games and hundreds of shipped students. You can learn more about Ian at his blog, Teaching Game Design.

March 2011: Using Metrics

Metrics.  For some, metrics is a golden savior:  the pathway to riches!  By learning what users respond to, you can quickly capitalize on this info and rocket your game to success.  Is it really that easy?  Surely, you need to know how to interpret the data so that you are not led astray.

When David Michael and I presented at the Serious Games Conference D.C., we explained that while it might be easy to mine user actions for data, it might be hard to tell if users had actually "learned" anything.  People who make surveys know that it's not always that easy to ask the right questions.  For one, are you even measuring what you want to measure?  Maybe the level of engagement is not shown in that indicator.  After all, this is just a bunch of numbers.  All subject to interpretation.

At this year's GDC, metrics certainly made an impression, but there were cautionary tales warning people not to get too beholden by them.  Metrics don't solve the world's problems.  They don't tell you WHY, just WHAT.

With this in mind, I'd like to know:

• Are you using metrics?  If so, why?
• What do you feel about using metrics and will it help the industry as a whole?
• How are you using metrics?  And how does this help you? 
• If we just cater to a certain audience's likes and dislikes, are we limiting our audience to a certain type?

March 2011 Poll

Please vote for the March 2011 topic! As always, feel free to suggest more topics!  Look at the submission guidelines for Topics and Blog Entries.

You'll see the poll to the right. The choices are:

• Using Metrics
• Gamification
• Game Accessibility 

Please vote by March 1.  Thank you!

February 2011: Cheats

What an odd day to be writing about Cheats!  Sorry.  Sorry for the delay and the unintended timing.

About a year ago, I prepared a presentation on the topic of Cheats for the NYU Game Theory Seminar Series.  It struck me that colloquially, cheats or codes, as we sometimes call them, are commonly accepted by players as part of the game.  Look up any game and the word 'cheats' on Google and there are sure to be a number of websites and videos on the topic.  There are even magazines solely devoted to game cheats.  What does it mean to cheat in a game?  How does that affect the ludic contract?

For certain, a lot of it is tied to marketing.  What used to be a shortcut is now collapsed into a code that gives you an item for buying a certain product.  You may get an in-game item that helps you or a way to skip through the roadblocks to get to an unlimited mode of play.

When I was working on Siege of Avalon, a single-player RPG, the designers did not care if the player chose to be unheroic and slay everyone in the castle instead of the orcs outside.  Clearly, that was unintended behavior, especially since every NPC you met hailed you as the mighty hero of the homeland.  We were also aware that players could jigger the numbers so that they were so powerful, so stealthy, so charismatic, etc that they could sneak up to mobs and slay them without any sort of effort but pressing the Attack button.  This was a single-player game, so such behavior did not affect anyone but the player.

In a multiplayer game, it certainly becomes more complicated and even more so when there are real money transactions for in-game currency.  There was a recent article about cheating on a Zynga game, so not even social games are exempt from this kind of behavior.  But even when there is no real money involved, just prestige and competition, players can have strong feelings about cheating.  In an article from last year, a player accused of cheating at Counterstrike was knifed through the head.  Clearly, there are levels of cheating that are considered acceptable by players and levels of cheating that are unacceptable.

So what are your thoughts?

• How does this affect the ludic contract?
• What is it about game culture that promotes the use of cheats?
• How does the use of cheats affect the game?
• What should be done about cheats? 

The Role of Crafting (Part II)

In Part I, game designer Brendon Trombley looks at the various ways crafting adds to player experience in MMOs and how developers can use an understanding of these roles to build better crafting systems.  In Part II, he discusses the role of crafting systems from a developer’s standpoint, which can differ and even conflict with the player’s agenda.

So what’s the role of crafting for developers?

It adds legitimacy to the game

It’s a sad fact that many developers include crafting in their game simply because they are expected to by the outside world. This leads to poorly designed systems that are added seemingly as an afterthought and can end up dragging the game down.

When designing in a genre where player retention is vitally important, developers should think carefully about their audience while considering crafting systems. If the game is meant to appeal to a specific player type, for instance casual or combat-oriented players, it may be possible to leave out crafting but include features that create similar benefits. Otherwise, if it’s decided that including crafting is indeed required, it should have the proper attention and budget allocated to it or players won’t use it, making it a wasted effort.


It gets players to spend more time playing

The goal of any system in an MMO should be to engage players and keep them willing to play the game over a long period of time. Crafting offers a form of character advancement that, when combined with combat advancement, creates plenty of motivation to spend time in the game.

When taken too far, this concept may lead to the addition of artificial time-sinks to crafting systems. Unnecessary grinding, long progress bars, and harsh penalties for failure may require the player to spend more time crafting, but at the cost of increased player frustration. Too much frustration, and there’s a chance they will stop the activity entirely. Avoid this by ensuring there’s always a sense of progress for the player. Perhaps the items they grind are ingredients for recipes down the line or components for other tradeskills. Perhaps advancement is based on experience rather than random skill-point increases.

It creates interdependency between players

MMOs, being multiplayer games, should of course include mechanics that foster player interaction and cooperation. Crafting systems are an effective way to do this. They allow players to create and enhance items for others, and promote the sharing of resources and materials.

A great way to create more cooperation between players is to include recipes with rare dropped components or ingredients crafted from other tradeskills. However, these recipes should be special exceptions that produce extra-useful items. Don’t require too much collaboration for basic items and especially not for regular advancement in the skill, or players will become frustrated at the extra time and money costs.

It promotes a strong economy

A strong economy increases player engagement and, if the developer so desires, can be a source of income if real-money trading of in-game currency is allowed. Crafting is a major source of trade between players. Additionally, it creates value in all those skins, fangs, gems, and other drops that would otherwise be merchant fodder. Merchant-bought components can help remove currency from a mostly positive-sum money system, reducing inflation.

Use crafting to promote the economy by first and foremost making the results of recipes worthwhile for all kinds of players. Then, create a variety of sources for ingredients: merchants, drops (rare and common), resource nodes, and other recipes. Create demand by designing ingredients to be useful in multiple recipes, and maximize the number of drops that are useful in at least one recipe.

Final thoughts

This is by no means a comprehensive list of all the ways to create a successful crafting system. Nor should all the tips here be included in the same game. Developers should instead prioritize their goals while thinking carefully about the goals of their audience. That way, they can craft a system that satisfies the needs of everyone as best they can, creating a stronger game in the end.

Brendon Trombley is a long-time player of MMOs and is currently a game designer for Quest to Learn, a school for digital kids in New York City that bases its curriculum around game design and systems thinking principles.