Monday, July 22, 2013

Can We Make Education Better, or Is This Really As Good As It Gets?

Adriaen van Ostade 007

I attended a summer conference years ago at Brown University. The conference theme had to do with making brain research more useful in practice. I was a doctoral student at the time, working in the emerging domain of Educational Neuroscience. The field was so new it wasn't even established yet, it was starting to generate some buzz, and this was one of the first conferences of its kind. I was pretty excited to be there.

One talk was by a cognitive neuroscientist renowned for his brain imaging studies of memory and perception (how we see, hear, etc.). This particular presentation, though, was about a study he was doing on learning. Learning and motivation have been my primary interests for a long time, and so I was keenly interested in the talk. During the Q&A following the presentation I asked, "Do you have ideas about how this kind of brain research might help us make education better?"

Based on prior experience, I expected one of two answers.

"Yes, I have several ideas about how this might be useful down the road.  For example..."

"No, this laboratory work is too preliminary and too far removed from educational practice for me to speculate about that."

His actual answer was far more interesting (and surprising).  He said:
"I think we do pretty well in education already. I don't really think it's possible to do much better."

I was taken aback. First, he dismissed my question instead of answering it. But, more importantly – could he really be suggesting that education is about as good as it can possibly be? And that a deeper understanding of the biology and psychology of learning won't lead to better education? At the time it hadn't occurred to me that people - especially people who study learning - could seriously hold that view.

But he got me thinking…

Why should anyone believe that education could be better than it is?  And how much room for improvement is there, really?

These are quite important questions.  After all, if the best-case scenario would be, say, 10% improvement over what we do now, then it probably wouldn’t be worth the extra cost.  But if, on the other hand, we could double the return on our educational investment (100% improvement), then that would really be something to get excited about!

How much more efficient could education be?
First, let’s think about educational efficiency.  One way to determine that is to compare how long it takes students to master a specific amount of content under different instructional designs. One telling example comes from the SHERLOCK Project, in which cognitive scientists demonstrated that they could replace four years of on-the-job training (in troubleshooting electronics) with 20-25 hours of computer-based training to produce the same expert-level performance. Four years of training compacted into less than three days through the systematic application of Learning Science.  The authors of this article point out that “SHERLOCK achieves this stunning result in two ways, by affording the opportunity for extensive practice and by creating educationally effective instructional conditions” (p. 9).

So, based on this example, we can estimate that in at least some domains education could be made not just 10% or even 100% more efficient than it currently is through optimal instructional design, but on the order of 100x – or 10,000% – more efficient!

Wow. I’d say that puts the score at:
Me: 1
Cognitive Neuroscientist: 0

How much more effective could education be?
Now let’s think about educational effectiveness. One way to determine that is to compare levels of student achievement in a subject area like math resulting from different instructional designs. The cognitive scientists behind the RightStart project did just that.  In one study, they followed three groups of children for about five years (from preschool through third grade), measuring their math achievement each year.

  • Group 1 was a high-resource group – children from middle- and upper-income homes who attended a magnet school with an enriched math curriculum (not specifically informed by Cognitive Science).
  • Group 2 was a high-ability group – children from lower-income homes who were identified as having superior math achievement when they arrived at school. They received no special or enriched math instruction.
  • Group 3 was the experimental (RightStart) group – typical children from lower-income homes with no special aptitude for math.  In addition to their regular school curriculum, these children received about 20 hours of intensive supplementary instruction in “number sense” (the conceptual foundations of math).  The special number sense curriculum was explicitly informed by Cognitive Science.

The RightStart group started out with the lowest math achievement of the three.  Over the course of five years, however, they reached higher levels of achievement than either the high-resource or the high-ability group.  Evidently, optimal instructional design based systematically on Learning Science can take the children who would normally sit at the bottom of the class and reliably put them beyond what is currently the top of the class – and they’ll stay there for years.

Wow again.  I’d say that brings the score to:
Me: 2
Cognitive neuroscientist: 0

Cognitive Neuroscience is not Education Science
Based on available evidence, it seems the Cognitive Neuroscientist was just plain wrong in saying that education today is about as good as it gets. But he’s a senior scientist studying learning – so where’s the disconnect?

I think Clarke’s first law applies here:
When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

What I have further come to understand in the decades since (thanks in no small part to this book by Dewey) is that there is a vast difference between a Cognitive Neuroscientist who studies learning and an Education Scientist who studies education. Much could be said about the differences between them. For now, I’ll simply point out that the former apply systematic methods of inquiry to describe neurological and psychological processes, while the latter apply systematic methods of inquiry to improve educational processes. The difference is analogous to that between the chemical sciences and the medical sciences.

I have spent much of the past two decades helping people produce the learning outcomes they desire (education) by applying systematic methods of inquiry (science). I am an Education Scientist. In retrospect, I think the Cognitive Neuroscientist was not wrong to dismiss my question. I think I was wrong to ask it of him - in the same way you wouldn't ask a chemist (even a brilliant one) to prescribe your medication. That's simply not his area of expertise.

Long experience in the field supported by evidence from programs like SHERLOCK and RightStart have led me to believe that we can make education better across the board. Not a little better - a whole lot better. Imagine, for example, that our schools could produce twice the learning in half the time at a fraction of current cost. I believe this is feasible today, in existing schools, with our current students, teachers, and facilities.

The way we can make education substantially better is with a straightforward three-step process. This process builds on the one applied successfully by the scientists behind both SHERLOCK and RightStart:

  • Base educational designs on scientific models of human learning.  There's over a century's worth of relevant research (including work in Cognitive Neuroscience) that is hardly being used in educational practice. We need to start using it. 
  • Use technology to dramatically expand access to high-quality learning experiences while lowering cost. Today, we could basically create for every learner a "personal-tutor-in-a-box." Students could work at their own pace on a personalized curriculum. The system could collect rich data on student performance. Teachers could spend more time actually teaching.
  • Stop arguing about what people think will work in education and start experimenting observing, acting, and sharing more systematically to determine what actually works. Simply apply scientific thinking to educational processes the same way we do to every other major domain of human endeavor like engineering, medicine, agriculture, and economics. 

Wash. Rinse. Repeat.

Today, we have all of the components we need to make education much better. There is definitely a way. The question is, do we have the will to make it happen?


  1. I don't think anyone objects to trying to improve our schools. The problem is convincing people of the right way to do it. The schools have been through countless new programs and most teachers I know are burnt out from the "new, new" method of teaching math. In essence, the schools and parents want proof before they try it. If you can prove that if we do X, we'll get Y then they will do it. Anything short of that and you are fighting an uphill battle

    1. > If you can prove that if we do X, we'll get Y then they will do it.

      But part of the puzzle is that this doesn't seem to be empirically true. In the early 1990's, for example, the RightStart researchers demonstrated quite clearly that if we teach kids number sense (according to their technical definition of that), then kids can succeed at math. There are studies from several independent research teams showing that number sense predicts success in 7th grade, high school, and in life (in terms of salaries after school, for example). If kids don't have number sense, they probably aren't going to succeed at math - which means they are locked out of STEM careers (Science, Technology, Engineering, and Mathematics).

      So here we have as clear a case that if we do X, we will get Y, and - a full thirty years after the demonstration - we still aren't doing it.

      > Anything short of that and you are fighting an uphill battle.

      I do agree with the spirit of this - that collecting evidence is necessary (though not sufficient) to move the process forward. But I also would argue that we can't "prove that it works" without running educational experiments - experiments in real schools, with real teachers and real students - to determine if it works. No lab experiment can tell us what will work in a real school. So we are going to need teachers, administrators, parents, and other stakeholders to partner in the process of proving that things work.

      This is another part of the Gordian Knot, I think - there's a chick-and-egg problem here (prove it works and we'll do it - but we need to work together to do it before we can prove it works...)

  2. This comment has been removed by the author.

  3. Your third point, to "stop arguing" and do something feels the most difficult. There is a gap between research and practice, that is not necessarily based on a lack of communication. To think of something rather simple--tons and tons of research and everyone knows what it says--such as teen sleep and school start time. Inertia, resistance to change, fear of disruption. Sometimes, on a discouraged day, it feels that we are at a place where it is as good as it gets because we can't get humans to get out of their own way.

    There is something peculiar about education as compared to other industries. Sleep and school start time, for example, has a handful of districts who have flipped the elementary and high school start times so that the bus requirements remain identical (no extra $$ incurred) and have fantastic results. Fewer ADD/ADHD diagnoses, fewer discipline probs, better test scores. One district in NJ documented that their average SAT scored went up 60 points with no other intervention. However, the human inertia around change--in this particular case, it is mostly about after school competitive sports--is the piece that confounds me.

    1. Based on my experience, I agree this is probably the hardest step, Stacie. I want to distinguish between two different facets of "research," though. There's clinical research (for example, on teen sleep) - that's biological research, not education research. Then people argue about what it means for educational practice - should we change school start time or not? My point is that the biological research on teen sleep does not - can not - answer the educational question about what school schedule to set. We need to run experiments and consider other variables not included in the teen sleep studies. Find a school willing and able to try a different schedule, and see whether it produces better educational outcomes, all else equal. Then, even if it works, we still have to ask the question whether communities are willing and able to make the change. A later school start time would be disruptive to work schedules, for example, which is no small issue to address. But having the data in hand about whether a later school start time actually does lead to better educational outcomes would at least enable us to have more informed conversations. And while it would require some effort and leadership, this is not an outrageous proposal to try in one school somewhere.

      In fact, I think it's important to note that this is precisely how virtually every other major (modern) domain of human endeavor operates and improves over time. Health care (medical trials), agriculture (agricultural science), marketing (A/B split tests via Google analytics), even business (for example, as seen in the Lean Startup and Growth Hacking movements). So what I'm proposing should really not be seen as radical - it should be standard operating procedure in education just as it is in these other domains. I agree that what's peculiar is that we don't do this as a matter of course in education already.

    2. I want to spend a moment more on this because you gave a good example, Stacie.

      Let's say that all of the following is true:
      * Physiological research shows that teens have an optimal sleep cycle that is at odds with the school schedule
      * Independent cognitive-behavioral research shows that better sleep means better learning
      * Based on these findings, a school decides to try out changing the school schedule so the teenagers can get more optimal sleep (if done systematically, this is education research)
      * The result of the education experiment is that students show fewer behavioral problems and higher test scores - a clear win in terms of learning

      So, on the question of "should we change the school schedule for teenagers?" what does education science tell us?

      Absolutely nothing.

      That's a value judgment - not decidable by science but to be decided by a community according to their values. And evidently, the community members where the school schedule experiment was actually done value preserving existing athletics opportunities more than they value the observed learning gains. While you and I might disagree with that value judgment, we can't rely on the science to back us up.

  4. I think experimenting in the schools can be difficult because the stakes feel so high. Parents don’t want their kids to be part of a group that is in a failed experiment because they’ll never get that year (or whatever amount of time) back again. The perception is that their kids will be at a disadvantage from that point forward. Even when some of the more “proven” methods are attempted the change can feel threatening. I know I’ve heard parents grumbling about schools teaching number sense, for example, because it’s not what they were taught. Instead parents want to know, “Why isn’t my kid memorizing the times table?” I’m not sure how, but in some way, parents needs to be convinced of the value of experimentation. Maybe if we all read your blog!

    1. Suzanne,
      This is a really important point. What parents don't seem to realize, though, is that everything teachers do is (or was, at one time) an experiment. Once upon a time, someone decided to try having kids memorize the times tables. At that moment, that was an experiment. And if it's never been systematically examined, then as far as I'm concerned it's still a large-scale, ongoing experiment that's stuck in the "pilot" data collection phase. If it wasn't an intentional experiment, it was ad hoc, which means it wasn't done systematically and its efficacy wasn't assessed systematically. So what parents should really be concerned about is that this kind of ad hoc, unsystematic experiment is being performed on their children all the time as the standard operating procedure.

      What's different about what I propose in the post above is not the experiment itself -- that is an unavoidable fact of life and goes on all the time. The difference is that I am proposing we acknowledge that this experimentation is going on and try to be more thoughtful, intentional, and systematic about it so that we can actually learn from it and do better over time. I think people anchor on doing things "the way I learned it" because in the absence of a rational process (educational science) there's an enormous amount of uncertainty and sticking to what is familiar seems to be a way of managing the worst case scenario. But in this case, at least, I don't think it is a way of managing the worst case scenario. The world these kids are inheriting is not the world their parents inherited. What got them here won't get their kids there.