In this MTP Engage Manchester talk, Lisa Vigar takes us through the journey – from launch proposition in September 2018, to winning a prestigious Webby award – of the BBC Voice for Kids Alexa app. She outlines some of the lessons learned around the challenges of developing voice products and working with children, and highlights the importance of detailed user research to engage and retain a regular audience (especially when your audience includes the complexity of parents and children!)
Misunderstood MVP
There is a saying that if you’re not embarrassed by the first version of your product, then you’ve launched too late. We all understand that the intent behind that saying is that you shouldn’t invest too heavily in an initial launch – an MVP – which might reveal you’re on the wrong track.
However, the first lesson the team on this product learned was that if you are truly embarrassed by the first version of your product, then there’s a good chance you’ve misunderstood what’s valuable to the audience, and don’t have a clear understanding of what they want.
To quickly test the appetite for a kid’s voice app, the team created one game but didn’t launch it due to lesson number two: if you’re embarrassed by the first version of your product, you’ve possibly misunderstand what’s valuable to both your audience and your business. The team recognised that to find the sweet spot, you need to understand what’s valuable to both.
It turned out that the business stakeholders weren’t comfortable signing off on such a stripped-down product experience, so lesson number two led to ‘MVP Take 2’. The first skills which were launched on the app were actually three games, followed a month later by three stories.
User Testing
The team did multiple rounds of user testing – exploring how the app was being received, and trying to learn what level of complexity kids can handle. It’s worth remembering that the business goal was to have their users (children) regularly returning to the app, and discovering the huge breadth of content available in the Cbeebies library. They quickly learned a few valuable lessons about presenting children with options for what they’d like to do:
- Children are generally capable of cognitively handling (making a meaningful choice between) just two options.
- Children tend to enthusiastically shout out answers before hearing the full question, so the team needed to invert the questions. Instead of asking “Who do you want to play with? Andy or Justin?” they would need to give the options first; “Andy and Justin are here – who do you want to play with?”.
The initial testing was insightful, but the initial wave of usage data was highly unexpected; the app was being used, but people weren’t coming back and listening regularly. Over 70% were only using the skill once a week, and quantitative data wasn’t telling them enough, so they started to dig into other sources of data:
- Skills store reviews – to see what feedback people were leaving
- In-home diary studies – they asked users to complete these in their home environment
From these data sources, they realised there were three key problems:
- Users didn’t know how much content there was
- Users only picked from the two options presented (they didn’t know how to go “off-menu”)
- Users are unable to find the content they want
How the team iterated the MVP
Considering that one of the team’s goals was to help more users to access the wealth of CBeebies content, this initial wave of usage data was troubling. Thankfully, they had the insights and the capability to dig in, iterate, and learn more.
Challenge Statements and Discovery
The team worked in challenge statements framed by “how might we…” questions so, based on the user insight above, they asked themselves:
How might we increase the breadth of content discovered and consumed by our audience?
This is all about Discovery, but discovery is very hard in the voice space with no imagery or physical interfaces! The next step was a workshop, bringing the multidisciplinary team together to look at research data, listening to the skill, and brainstorming – they came up with lots of post-its and ideas.
Prioritisation
Pulling together everything they generated, the team went on to do a traditional impact/effort analysis to prioritise the top three ideas:
How might we…
- Tell users how much content we have
- Allow users to “scroll” through our content
- Support any content title (brand or character)
Taking what they learned into consideration, and thinking about the problems they were trying to solve, they also developed the following design principles:
- It must make sense that only two options are presented
- If there is a question, end the sentence with it
- Be as precise as possible
While having those requirements was a great step forward, trying to solve for all of them simultaneously felt impossible. That was, until the team realised they could solve the problem by completely removing the “question” component of the voice interface:
- They could tell the children how much content there was to play with, introduce the scrolling functionality (choose one or ask for more),
- support any title without tying the content to a single character,
- and lastly, be suggestive by asking “how about this option”, instead of “choose A or B”.
The navigation became instructional without a question, and this was all done in 18 words. The team tested the functionality with a group of children using a lo-fi prototype and it worked incredibly well. The children said “more” a lot, and both frequency and retention increased, although interestingly more from the stories and songs than the games.
If you’re curious about where this commitment to research and iteration led, as of February 2020 (the time of this talk), the BBC Kids Skills App has won awards, has five games and 31 stories, and the team continued to release new content every month including song playlists.
