March 2011: Using Metrics

Metrics.  For some, metrics is a golden savior:  the pathway to riches!  By learning what users respond to, you can quickly capitalize on this info and rocket your game to success.  Is it really that easy?  Surely, you need to know how to interpret the data so that you are not led astray.

When David Michael and I presented at the Serious Games Conference D.C., we explained that while it might be easy to mine user actions for data, it might be hard to tell if users had actually "learned" anything.  People who make surveys know that it's not always that easy to ask the right questions.  For one, are you even measuring what you want to measure?  Maybe the level of engagement is not shown in that indicator.  After all, this is just a bunch of numbers.  All subject to interpretation.

At this year's GDC, metrics certainly made an impression, but there were cautionary tales warning people not to get too beholden by them.  Metrics don't solve the world's problems.  They don't tell you WHY, just WHAT.

With this in mind, I'd like to know:

• Are you using metrics?  If so, why?
• What do you feel about using metrics and will it help the industry as a whole?
• How are you using metrics?  And how does this help you? 
• If we just cater to a certain audience's likes and dislikes, are we limiting our audience to a certain type?