Always Be Playtesting

People are often surprised to hear that I am already running playtests of my prototype with external people on a weekly basis. After all, Conjure Strike has only been in full development mode for a bit over a month at this point. Especially in the independent and small studio game development, games can often seem like an extension of your own self worth. By pushing something that is crude and buggy, you are opening yourself up to criticism that may be painful and demoralizing. Games are also especially difficult to do user research on because in order to have something fun, you often have to have multiple completely systems (Win Conditions, Health, UI/UX, etc).

My product management background taught me to never shy away from user testing. In my last “real job” the product my team was working on a product that services users located in India, Indonesia and other developing nations which resulted in me personally spending about 3 months in India and some time in Indonesia conducting user testing with my team. The skills I learned working on user testing have translated well to game development.

My philosophy stems from the concept outlined by Steve Blank and Eric Ries — Lean Startup and Customer Development. It is a process in which you must assume that even for the best entrepreneurs, the majority of your hypotheses and ideas will end up being wrong. If you wait until the end before you are able to verify your hypothesis then it will already be too late to make changes. Whether you are working on a passion project or starting your own game company, the thought process should remain the same: you can’t know whether something is good until you test it, and the longer it takes for you to test something the more uncertainty lies in the probability of your success.

In testing Conjure Strike and it’s predecessors, I’ve heard a huge range of feedback from “It’s not quite there yet, but it’s got potential” to “This isn’t actually your game right? Is this a joke?”

Hearing things like that make me cringe, but as a indy you should embrace it. Dig deeper and find out what the tester is actually trying to tell you! Is it that the locomotion system feels off and they are getting sick? It’s our natural reaction to shy away from this kind of feedback, but imagine how bad you would feel if come release day you receive the exact same feedback, too late to correct?

More recently I started hearing people trash talk when they won and curse the heavens when they lost. I had testers have so much fun that they hit me up the next day telling me how much potential they see in our game. The good moments come in trickles, but I’ve come to cherish those moments. Those moments would not have been possible without the constructive feedback from nonplussed testers in previous sessions.

I’ve been fortunate to work with brilliant user researchers over the years to learn how important it is to have a process and schedule put in place for user testing

Here are some practical tips to maximize your user research output:

  • Have a plan – an introduction that puts the user at ease is important. This is not an intelligence test for them. Make it clear that if they can’t figure anything out that’s not their fault, it’s yours.
  • Understand the source of feedback. Your best friend saying “I think it’s pretty fun” non committally is not necessarily a positive comment. Survey data shouldn’t be used alone to make product decisions.
  • Listen more than you talk, and try to get the person to speak.
  • Integrate user testing into a normal part of your development cycle. For Conjure Strike I am trying to have the next 2 weeks of user tests lined up always
  • Be ready to act on feedback. Don’t just test and ignore it
  • Don’t be defensive, don’t suggest ideas to improve something on the fly. Simply accept the feedback and solution later.

I will be happy to help you with any User Testing, or Product Management questions you might have. Feel free to reach out.