Why personalized learning keeps failing
In my inbox sat a New York Times article sent by my girlfriend’s mother: Silicon Valley Came to Kansas Schools. That Started a Rebellion. The article tells the story of a 8,000 person town of Wellington all but lighting the school on fire after the Chan and Zuckerberg Foundation’s personalized learning software was introduced; software that was developed alongside $99.1 Million dollars of grants. “Isn’t that what you do Elliott?” asked Natalie (girlfriend). I write this response article in part to save relationship face, but also to bring to light why this outcome is not surprising at all from my research.
Take a deep breath, it is going to be a bumpy ride. As I tell the edtech companies I advise, “First off, you’re not alone. I have watched hundreds of hours of elementary school kids use digital software, and I have yet to see one child learn a new math concept.” For the last five years, I observed Boys and Girls Club students fail at using educational software (with help from eye tracking and emotion sensors). A child will listen to a song or mini lecture about finding the area of a triangle for the first time. But afterwards, they consistently fail at solving that area. And then they give up and think math is boring.
Personalized Education does not mean Individual Education
When we translate our educational theories to the digital world they falter. Most educational editors are copying what they did as a child: an adult tells you what to do and then you repeat. But these digital designers do not include all of the subtle things a teacher does. Children care a lot about their teacher so they are fixated on her. But when educational videos come up, children’s eyes consistently drift away, looking for other, more emotional stimuli. A teacher gives feedback and understands when the child is confused — teaching is more of a dialog than a lecture. Computers are agnostic to where the child is at.
The Boys and Girls Clubs of Denver have a reading room where the children can choose to read after school. Daily, children line up at my research station and ask if I would read to them. One of the most impactful things a volunteer can do at the club is read with a child. Sit with them. Ask them questions. Sound out words together. Learning is social.
ThinkCERCA, capitalizes on this: students come together as a group and have discussions on articles they read on the computer. The whole second grade class debates whether we should get rid of roads to save animals. You can see the students thinking and reacting to one another, applying what they read.
In our prototypes, sometimes we let students work together, despite our client’s request to avoid social: they have to work alone. When children are allowed to work together, their learning is amplified; they come alive. Children laugh at jokes together, support one another in hard problems, and celebrate shared successes. I wish we would design software that was meant to be used as a group.
One Summit Learning parent described the Summit software as, “We’re allowing the computers to teach and the kids all looked like zombies.” I believe this is a byproduct of how we view computers and software: PC’s are designed to be “personal”, for the individual. You do not see a lot of laptops designed for three mice. In one school I visited, the most social interaction students received was by logging out of the educational program and sharing YouTube videos with one another. Teachers check their email for the entirety of the computer lab hour instead of checking in with students (this is with me and a camera watching them). The New York Times articles stated that teachers would only check in with students two minutes a week. I am not surprised, the Summit program was designed to not include the teacher in learning.
At first I blamed the teachers for this siloness. But now I have come to understand how these programs are designed. It feels rude to peep over a student’s shoulder. No questions say “Ask the teacher or a friend…” In an effort to make things more independent and easy, the programs are designed to boot classmates and teachers out of the experience. Those same classmates and teachers who are the foundation of strong learning.
Suggestion #1 to Summit Learning: Like Facebook knows, humans are naturally social. Innovate new ways to make digital learning a social experience where we are learning together: at our own pace, but together. Recognize that this will be hard. Recognize that the default strategy of the software, students, and teachers will be to silo up, despite no one wanting that. Design against this natural tendency, design your software to be intentionally social.
Treat Science as Ideas, not Truths
In the New York Times article, John Payne of the RAND Cooperation states “There has not been enough research.” But I think research is the wrong word. For the most part, educational theories about how to teach the area of triangles digitally has failed students. I already have plenty of video evidence of disgruntled students not learning and now Summit Learning has rebelling schools as a clear indicator of failure. We need to move past validitive research: spending a large amount of resources to prove old methods are not working well. Instead, the Chan and Zuckerberg Foundation should invest in generative research that looks to innovate new ways of teaching digitally. The new research should be messy, trying new things, making mistakes, and starting in classrooms with students who would might someday use the software. Educational research should feel more like a hackathon than a standardized test.
I tell my clients academic research is simply a pool of ideas. Do you want to teach with a growth mindset angle, a constructivist foundation, or a more didatic method? All of those ideas are fine, but they should all be considered only as potential ideas, nothing more. Take those years of research at Stanford, build something out of it in two days, and test it for a week at a school. In that one week you will learn more about what is effective in your unique environment than those ten years of research.
In a recent project, I was asked to better engage middle school students with taking tests. I copy and pasted four different methods (like Growth Mindset) and gave them to students at the Boys and Girls Club before they took a test. All four digital methods crashed and burned for the students, with students quitting my math test after about 3 to 5 minutes.
I then co-created with the students: they showed how their peers motivated them more than a computer program. When a high school student with your same skin color talks, that means something. I do not know what academic theory the students were tapping into, but that was the winning idea. We created videos of high school students telling how they double checked their answers to pass. It worked, students wanted to take the test.
Suggestion #2 to Summit Learning: It is good you are looking at science, but do not treat that science as god spoken. Your classroom is not 18 year old white males in college, where the vast majority of academic research came from. Instead, prototype and try out a diversity of theories until you find what works for the children and teachers you serve.
Personalized Learning is Missing Human Centered Design
“Good design copies, great design steals.” — Pablo Picaso
One of the largest challenges that personalized learning faces is that there is not a clear working model to copy from. I cannot point at a piece of educational software and say just do that. And because there is no working method to copy, designers of personalized learning tools need to recognize they are taking stabs in the dark.
To make matters worse, kids are often a voiceless consumer. My clients seldom ask children what they think or what is broken. If you take time to observe and listen to what kids want, they will paint a different world for how digital education should look like. The same goes for teachers — they have needs that are going unmet too. As long as funding and permission comes from school boards and admins, edtech companies are likely to miss their real consumers. When you fail to listen to your customers, your students are going to start dropping out of schools and teachers are going to post protest signs. It is your fault they feel this way, not theirs.
In the New York Times article, the Summit team came off as distant — claiming the parents and students are nostalgic and resistant to change. A more human centered approach would be to listen and embrace these complaints. If a child does not like your program, there is something there for you to learn and improve on. I do not know how true it is that the Summit team is not human centered (as the article suggests); my peers in the field have said otherwise. None the less, there is a lesson here about the importance of putting students and teachers first.
I understand why a human centered design process is hard. When I spoke with the Charter School Growth Fund, I asked how they knew their charter model would work. We had a heart to heart about how hard it is to get permission from all the stakeholders and funders to experiment. As an innovator, you are diving in, hoping all the pieces work on the first try. They seldom do. I advised the fund that they should instead take one class and rapidly iterate on the experience — make it work the way they wanted it to, before replicating to other schools. This process would involve daily iterations and heavy co-creation with the teachers and students. They agreed, but that is not how funding works — you have to sell a perfectly baked solution from the get go.
Suggestion #3 to Summit Learning: Tackling personalized learning is a monumental design task. Give it the time, space, and resources to nourish. You started out by jumping into a school system, but your idea was not fully fleshed out. What would have happened if instead, you tried it in one class in Kansas school and solicited feedback? When students told you they felt like zombies, you would have time to respond. You would have seen teachers failing to interact with students, and could have made changes. Human centered design is not just a mindset, it is an environment. Create the space where mistakes can be made and addressed without children dropping out of school.
Keep Going
The New York Times article all but said personalized learning is a failure. I used to think the same. However, after years in the digital classroom, I see potential, potential that keeps me writing. Personalized learning allows students to:
- Work at their own pace. If they figure out the area of a triangle, they can move on.
- Dynamically move in a lesson depending on their needs. If they forget what area means while learning the area of the triangle, they can move back to finding the area of a square first.
- Learn by doing. Children learn by trying out things, making mistakes, and getting feedback. Personalized learning provides a future where they can play and learn at the same time.
But for today, these promises still remain at a distance. These features are difficult to implement in a non ideal system and require deep and intentional user research and design thinking. Hence why these are more opportunities than realities.
I do not personally know the Summit Learning software (I could not log in or try the program out). And I do not know how much of the New York Times article was true or slanted. But all of the issues that the article cited are the same as what my other personalized learning clients experience too. The digital education problem is not a Chan and Zuckerberg Foundation problem. It is a design problem affecting all of education.
I believe with strong human centered design and strong research, Summit Learning has the potential to reinvent how we all learn. Here’s hoping they learn from Wellington and do.
