Tricks For User Testing With Kids

Often times when desigers or artists come to us researchers with a new build of their game, a still raw cinematic, or a couple versions of design, they say: “ We want to test if kids would like it.” If I ask them do you have a rough idea how do you want to approach, most common answer I get is “we can just ask them do you like it” or “let’s have the kids pick which one do they like the best?”. Nope! I always have them take those two questions out of their interview script.

Why? Because I can already predict what the kids would answer: “I like it!” or even “I LOVE it!” with a super innocent smile on their face. They know exactly that’s what adults want to hear, it’s a safe answer.

If we ask them to pick a favorite, it’s just not fair for the kids because the designs are so subtly differenciated — I mean, basically, the artists or designers can not make a decision by themselves, that’s why they want the kids’ help them. That’s why we constantly saw that the kid would pick one, then they will point to the other one “I like this one too…”, and the other one “Oh, this one too.”. If they got pushed too hard, some might even give up — “ I don’t know!”. Again, it’s just unfair for kids. Designers know there are good and bad mixed in each solution, but they can’t tell what are good and what are bad, so they throw the whole mix to users, hoping to get a quick and easy response. Well… good luck on that!

How would I suggest to test it in this situation? Here’s the trick #1: Decompose your testing item to a number of design elements, then ask questions individually:

What do you like about this video? Why?

What would you do to make this video even better? Can you use your imagination here for me please?

Trick #2: Take a detour, ask questions indirectly to get what you want:

- Would you like to play this game again in the future?

(Instead “do you enjoy playing it?”, but with a greater chance to get honest answer.)

- Can you tell me what happened in this video?

(What they remembered, what they like to talk most about are the good parts)

- Do you think if your friends at school would enjoy playing this game?

(It’s the same question as “do you like this game?” but guaranteed you get less lies)

- Do you think if your little brother would like to play this game?

(This could be good to find out how the game’s difficulty perceived. Kids more often would say “Too easy” if you ask them if this game is too easy, too hard or just right, who doesn’t want to be the smart one! I don’t need to tell you how many times I hear kids say it’s too easy even I just witnessed the entire struggling).

Intuition grow in practice. When I found myself 7 out of 10 times can somehow predict kids’ answers and reactions to the testing items,I started to ask myself and other UX designers: What’s the point of testing if you already know what answers. But wait a minute, let’s flip the hand, can I still trust the answers I get from my participants and rely on them for my design iterations? That’s when we need to rethink the testing methods when facing unique audience.

Nothing complicated, I usually take the three thinking steps:

1. What is the testing goal here? Yes producers will tell you simply as “We just want to know if kids like this”, but from a design perspective, you need to dig into this and find out what are the debates on certain design elements. There are A design and B design, what separate them out? What is the context behind those variations? Are we looking at the graphics design or the interaction with this icon we are talking about? Is it the art style or the animation transition that we are not sure about?

Always try to be more specific and focused.

2. What questions should we ask? Just like you wouldn’t ask a landscaper to fix your computer, you shouldn’t throw inappropriate questions to kids either. They are very special in almost every perspective, their age, cognitive load, motor skills, expressions… That’s really what makes this job challenging but rewarding.

Here are a couple examples. In a test, the kid just told you 4 things he likes about design A and 1 thing about design B, do you still want to ask him “which one do you like better?” even it’s written in the script?

Curriculum specialists want to see if the game is learning effective. They want you to ask a 4 year old “What did you learn from this game?” If you are lucky, you might get “math” “numbers” “letters” etc.

What else do you expect for a 4-year-old to summarize an educational conclusion from a video game?

3. How should we phrase those questions? Yes, yes, when testing with kids, of course the moderator should speak the kid’s language. But there’s more. You also need to understand how does their brain work behind those words. Typically, what I learned is that building the trust is the key for the first step. Then there’s a tricky balance to maintain between being the buddy and being the moderator, being friendly but not your friend (yet). I talked some more examples of getting honest feedback from kids through a better phrased communication.In another of my writing <Are The Young Testers Lying To You>,

Tailoring your language to children at different stage in their early education is also extremely important. Kids generally don’t want to be considered younger than their age because all they want is to grow faster and faster, especially those who started 1st Grade, you seriously need to treat them like big boys and big girls. Yet, at the same time, you don’t want to be too philosophical or obscure. Always prepare to explain something in a super simple way with super simple vocabularies.

Children are art, research is also art. When two arts intertwine, you see spark, spark and sparks.