How do children reformulate questions?

Research insights and design recommendations for speech interaction.

Photo by Colin Maynard on Unsplash

The most annoying experience with voice user interfaces is when they don’t understand what you are saying. What’s worse is when they don’t give you any clue on what went wrong.

Whenever there’s a failed exchange of information, virtual assistants would often say, “Sorry, I don’t know.” The burden of repairing communication is passed on to the user, as affirmed in a recent ethnographic study. In the said study, researchers documented how adults apply several strategies to make successful interactions with their smart speakers.

If adults are having difficulties, what more can we expect with children?

Motivation for designing speech interaction for children

If you think that speech interaction is still emerging, then consider this: 81 out of the 87 participants (aged 5 to 12 years old) in a study in Minnesota, USA have used speech interaction before. The children either have their own smartphone or borrow their parents’. Others have Google Home or Amazon Echo installed in their homes.

Virtual assistants and smart speakers have become commonplace; one user group we shouldn’t forget are children. Speech…

Children’s question reformulation strategies

Yarosh and colleagues conducted a study to find out how children deal with communication breakdowns with virtual agents. When the virtual agent prompted the children to ask their question in a different way, the children resorted to the following communication repair strategies:

Repeating the question — increasing volume, emphasizing words, and exaggerating pronunciations without changing the word structure of the question.

Reordering words — reordering words and phrases without adding new information nor removing complexity.

Substituting words — replacing a word or phrase without adding new information nor removing any complexity.

Expanding pronouns — replacing pronouns in a question with the corresponding noun.

Stating context — adding new information before asking a question but phrasing it as separate keywords or statements.

Adding context phrases — integrating new information to the question effectively narrowing to the case of interest.

Going off-course — asking a different question.

Most children tried multiple strategies but many were stuck with a single strategy until the researchers gave them a hint. This implies that in home settings, adults would still need to supervise and support children when using speech interfaces.

The most unsuccessful reformulation was repeating the question without any changes. Study participants assume that the question was misheard, so they simply repeated the question with increased volume or modified pronunciation, which is consistent with the findings of another study.

The most common reformulation that brought the children closer to a successful reformulation was expanding pronouns and adding context phrases.

Supporting children’s reformulation strategies

Based on the reformulation strategies and the hints that worked with children, researchers made the following recommendations. Designers and developers need to use “Sorry, I don’t know.” less frequently and handle errors better by doing the following:

Restate what was heard — Children often assume that the speech interface mishears their question so they focus on their volume, speed, intonation, and pronunciation. If the system gives feedback that it heard the question perfectly but still couldn’t find an answer, then children might proceed to use other strategies like changing the semantic or the structure of the question.

Ask for missing information — If only a piece of context is missing, the system should give a clarification response asking only for that particular information. For example, if a system responds with “What is ‘it’ in your question?” then children can proceed with disambiguation by expanding the pronoun. Asking only for the missing information is a good way to show system status in voice user interfaces.

Communicate limitations — If a particular type of question is difficult for the system, communicate this to the user and give them alternatives. For example, the system can say, “Comparisons are hard for me. Can you ask about each part of your question separately?” In other words, systems should clarify what it can do and how well it can do it, which is part of Microsoft’s design guidelines for human-AI interaction.

In summary, children use seven communication repair strategies:

1. Repeating the question
2. Reordering words
3. Substituting words
4. Expanding pronouns
5. Stating context
6. Adding context phrases
7. Going off-course

To support these strategies better, systems should:

1. Restate what was heard
2. Ask for missing information
3. Communicate limitations

These findings are consistent with existing guidelines for speech interaction and artificial intelligence systems.