We’ve got your back with this portable card deck of awesome testing questions.
I made this because:
#1 The most insightful questions are counter-intuitive.
For many of us, our intuition is to ask what we’re trying to find out — for instance, “is it fun?” or “do you like it?”. But your test goal is not the same as your test question. (I learnt this from UX researcher Erika Hall) Why? There are a few reasons. One being your playtester can try to please you rather than tell you how they really feel. Another is that terms are interpreted differently. What is ‘fun’ for them is different to you. Another is that our brains don’t always neatly categorise an experience in the same way. Would they call what they just felt, fun? But there are ways to get around this. For instance, you can ask,“How much time did you feel like you were playing for?” instead. Why? Because if they’re enjoying themselves time goes quickly. If they were confused or frustrated, it will feel longer.
#2 Status quo design does not consider the social context of testing.
Status quo culture encourages certain attitudes and assumptions about works-in-progress, giving and receiving feedback, designs, and designers. The biggest assumption is that truthful feedback is painful. Nope. Design is pretty cool and life changes. You can design your testing sessions so they’re both insightful and joyful. Insights that make you excited to keep developing are a true signal. We get to this by considering the actual design context:
- Testers can interpret certain questions as cues to please you or to hide their own insecurities.
- Testers can default to what is wrong without giving insights into what is working.
- Testers can view anything that doesn’t match their expectations as a universal gap or problem, rather than their subjective view or need.
- Testers can assume that anything that needs work or is missing is a signal of a lack of skill or knowledge, rather than what happens when the creative is focusing their attention elsewhere on solving new challenges.
- Testers, especially fellow developers, can default to trying to fix your game/project without finding out what you’re trying make or how you’re making it.
- Fellow developers can assume that the design choices that are invisible to them are the absence of design.
- Feedback sessions can leave you feeling deflated and further away from your goal, when they can easily inspire you into action.
- Assumption that games/projects are just executed rather than going through periods of figuring it out!
- Being let down by team members and having to test with a broken prototype.
- General discomfort people have with public learning, not-knowing, changing their mind, and so on.
- Tester feedback that skips what is a huge motivator to the developer: whether the tester having a meaningful experience (whatever that is to them). (I learnt this from Liz Lerman)
- Developers often running test sessions by themselves with little assistance (see access needs).
#3 Testing happens in the context of diverse access needs.
- Developers experience delayed processing, overwhelm, and exhaustion after taking in lots of information from testing, especially over hours, days, and in busy environments.
- Developers can feel unsafe around biased or judgemental testers.
- Developers can put a great deal of effort into the prototype but run out of time to prepare test questions.
- Developers experience memory differently. They may know great questions to ask, but can forget them in the heat of the moment.
- Testers can find questions communicated through various modes helpful: verbal or visual, or both.
- Conflicting access needs. The developer’s need to learn by trial and error can conflict with a tester’s need to have cognitive load reduced with more refined prototypes.
I’ve experienced all these epiphanies and problems. Over the past couple of decades of testing digital and tabletop games, VR, installations, ARGs, live events, literature, films, and plays, I’ve learned awesome questions to ask. Questions that give me great insights, intervene in some of the bias bullshit, and get me excitedly back to creating. I learnt the questions from fellow creatives, research studies, and I’ve developed my own. I’ve taught them to creatives at universities, gave a talk at TGDA’s DEVCON last year (the insights of which have been used at an education institution), and just submitted a book chapter on the subject. I’ve converted this learning into a handy deck to have at hand, and put them in a tin to make sure the cards can live in my bag.
Playtesting Buddy is the system I need, and I’m not the only one. Create one for yourself, or utilise the one I’ve made:
– Hardy tin case to handle rocking around in your bag.
– Rounded small (business-card) size cards so they don’t take up table real estate.
– 50+ questions to choose from.
– Questions for before and after testing, as well as ways to be great tester for others.
– High-contrast white text on a darker, plain background, with a large font.
– Accompanying guide explaining the reasoning behind the questions, so you understand the why and can develop your own.
– Blank cards to add your own. (And let us know so we can update the set!)
– Available print-on-demand delivery to many countries around the world (all except Brazil and Belarus — but we can post to you instead!).
Coming soon!