*think will work*in education and start experimenting to determine

*what actually works*.”

There were a number of reader comments (thanks to Jeff, Stacie, and Suzanne in particular) that helped me to understand that the word “experiment” may carry some extra baggage that wasn't part of what I had in mind here. In this post, I’ll try to clarify what I mean and offer an example.

###
How __not__ to run education experiments

First, a few things I

*don’t*mean when I talk about “education experiments”:

- I don’t mean we should bring scientists in white lab coats into the classroom so that they can better understand how our children’s brains work – that is
*cognitive*science, not*education*science. - I don’t mean we should bring academics in white lab coats into the classroom to test their speculative or radical new math curriculum for the very first time on our children for the benefit of future generations.

- I am more interested in research that has a good chance of benefiting the
*participating students*in the short term, and - I am interested in research that leverages the
*teacher’s special knowledge and insight*about the capabilities and needs of*those particular students*.

### A different take on education experiments (with an example)

What I am proposing is more akin to action research, or design-based research. To clarify what I have in mind, I’ll give an example from real life.

In my work as an educational designer and consultant, I have the pleasure of talking with some extremely dedicated teachers. Sometimes I meet them when they send a message, out of the blue, with a specific idea or question about how to do more for their students. Below is an example of one such message I received about a year ago.

Key points to note about the situation:

- This teacher has identified a
**problem**:*Something isn’t working right, and my students are struggling in math.* - This teacher has a
**hypothesis**about the root cause of the problem:*I believe some of my students lack number sense, which prevents them from understanding the more complex material in the math curriculum.* - This teacher has done some research (most likely on her own time) and identified an
**option**that she believes could address the root cause better than the current curriculum alone:*I have found an iPad math curriculum that I believe will solve the problem by developing my students’ number sense.* - This teacher has encountered a
**barrier**that prevents her from trying the option:*I don’t have all the technology I need (wifi) to use the app in my classroom.* - This teacher is actively seeking creative ways to
**overcome the barrier**:*Is there a way I can use the app-based curriculum without wifi?*(In my experience, teachers are willing to go to great lengths to make something work if they believe it will help their students.)

Here are two common outcomes:

__Either__: The teacher gets no support to overcome the final barrier, she can’t even try out the option she believes will help her students, and her students continue to struggle with math – probably for life (because math__mis__understanding – like math understanding – is cumulative). Sadly, this is probably what happens to proactive teachers and their students most of the time. The teacher takes the ball to the 99 yard line and for any number of reasons can’t carry it the final yard alone so it sits there. The students miss out. The dedicated teacher gets a little more burnt out – maybe this is even the last straw and she leaves the profession. Everyone loses.__Or__: The teacher finds a creative way around the barrier, tries out the math app with her students this year, and based on her results she decides whether to use it again (or try something else) with her class next year. The teacher goes to great lengths to get her students more of what they need. Her twenty-five students benefit. She feels empowered - maybe she can make a difference through diligence, resourcefulness, and a lot of hard work. But…did her idea work? How well? What’s the evidence that it did or didn't? Did it work for some kids and not others? If so, which ones? What did she do to make it work? That teacher has a lot of valuable, actionable insight as a result of her experiment, but it was a private experiment and so the world will never know about it.

It’s déjà vu all over again.

Key points to note about this situation:

- This teacher has
__independently__identified the same**problem**:*Something isn’t working right, and my students are struggling in math.* - This teacher has
__independently__produced the same**hypothesis**about the root cause of the problem:*I believe some of my students lack number sense, which prevents them from understanding the more complex material in the math curriculum.* - This teacher has done some research (most likely on her own time) and
__independently__identified the same**option**that she believes could address the root cause better than the current curriculum alone:*I have found an iPad math curriculum that I believe will solve the problem by developing my students’ number sense.* - This teacher has encountered a slightly different
**barrier**that prevents her from trying the option:*I don’t have all the technology I need (iPads) to use the app in my classroom.* - This teacher is also actively seeking creative ways to
**overcome the barrier**:*Is there a way I can adapt the app to a whole-class format so I can make it work with the technology I do have available?*

- If these two teachers face the exact same problem with teaching math, how many others are there? Tens of thousands? Hundreds of thousands?
- How much teacher time, energy, and good will are we wasting with all of this duplicated effort?
- How many
*more*teachers would use an effective option that had already been vetted by another teacher if they knew about it and didn’t have to re-invent it from scratch or take the risk of "being first"? - Looking beyond first grade math, how many students would benefit if every teacher didn’t have to reinvent the wheel like this on every big and little problem they encounter in the classroom? Presumably
*every single student*. That includes your children, and mine.

*This is going on all the time. Millions of such private experiments are presumably being conducted by teachers every day in classrooms across the country and around the globe. The only thing that’s missing to capture the value of that activity is a bit of systematic record keeping and a way to share results.*

### Where do we go from here?

I hope that helps clarify what I meant when I said we need to stop arguing about what people*think will work*in education and start experimenting to determine

*what actually works*.

In fact, I now realize that I mis-spoke when I first said that. We don't need to

*start*

*experimenting*- that part is already happening. As a first step we simply need to

*start*

*sharing*what teachers have discovered about what works and doesn't work as they try – sometimes desperately and at significant personal cost – to give our children more of what they need to succeed in school and in life.

I’ve read your last three posts with interest. You, and those who replied, raised many important questions. I’ve been mulling these over wanting to reply, but did not know exactly where I wanted to weigh in first. I think this is a good spot.

ReplyDeleteI can relate to the two teachers you cited as I am a first grade teacher and I too wrestled with why my students struggled with number sense (as well as issues of curriculum and pacing). Independently, I followed (basically) the outlines you gave as examples of how scientific experimentation happens every day in real time. I even attempted to replicate this investigation with our kindergarten classes, but was unable to do so for a number of reasons. So, I am one teacher, with one positive experience, and no real way to systematically analyze and disseminate what I learned so that others may benefit.

I believe there are solutions to this problem, just as there are solutions to most problems, but we have not hit upon them as yet. Or have we? Some educational content areas do a better job than others when it comes to scientific experimentation, sharing of results, and replication: elementary literacy for example. Math, it seems, is the content area that has received the least amount of real time classroom research.

I agree sharing is a first critical step. Unfortunately, many times this sharing does not even happen between grade level teams, much less across districts and outward. I wonder why? I think the answer is complex and we need to spend some time looking at this to determine the multiple (possible) reasons, and multiple contexts in which this occurs, discuss possible solutions, find models where sharing works, and then test each of these. Sounds strangely familiar! It might seem like we would end up back at square one with the question of how do we share results, but the idea is that our experimentation would have led us to that (those) answers(s).

So, let’s start with this: where have you found actual models of sharing that work? (By “work” I mean information is diffused at scale, models are replicated and tested, and the cycle continues.) Let’s use this space as a sort of huge white board where we can brainstorm ideas, “write” them down, cross them off, try again...and actually, actively, work to find solutions.

Rene,

DeleteThanks for sharing your experience. I feel strongly that we should support teachers like you in doing that, and that it shouldn't be nearly as difficult as it is.

> Unfortunately, many times this sharing does not even happen between grade level teams, much less across districts and outward.

Yes - this is somewhat puzzling. Teachers in other countries, like Singapore and Japan, share much more and routinely produce better outcomes. (So these are two places we can find successful models of teacher sharing.) One challenge is that the conditions are different across countries. In many other countries, teachers have a lighter teaching load than U.S. teachers because there is a recognition that they need more time for preparation during the school day. That makes it easier to collaborate with peers.

Maybe we can start by looking there. When do you prep, Rene? How does your prep schedule match up with the other teachers in your school, district, extended network? What times throughout the school day or week would you have time to share ideas, insights, and questions with your colleagues if everyone were inclined to do that?

P.S. LOVE the "huge whiteboard" idea!

DeleteIt seems to me that sharing the results of these "experiments" would be wonderful. It also seems that this is a problem that happens throughout society, not just in schools. How many employees figure out better ways to solve work issues but this solution never gets disseminated to other employees?

ReplyDeleteMoreover, if each teacher is running experiments, wouldn't it be difficult to figure out which experiment had the best long term effect? By 4th grade the student would have been involved in 5 different teacher's experiments. Where is the control?

Great point, Jeff.

DeleteIn the two examples cited in the post above, one decent control is the previous year's standardized test scores. Disappointing scores are frequently the reason teachers start looking for better tools in the first place. It's not a perfect control, but then again we don't need perfect (can't afford it, in fact). We just need to keep getting better over time.