Thursday, February 20, 2014

Khan Academy: The Illusion of Understanding (Part 1)


Guest blog by Dr. Marc Schwartz
Professor of Education at the University of Texas at Arlington
Director of the Southwest Center for Mind, Brain, and Education
This post is based on an article by the same name published in the Journal of Asynchronous Learning Networks.

The Illusion of Understanding
For the past three decades I’ve been working to dispel myself of an illusion that’s hard to recognize and even harder to overcome. I call it the “Illusion of Understanding.”  It’s the false belief that we understand something but then we discover we actually don’t.  The example problem below will help clarify what I mean.  Consider the following…

Iceberg Challenge
Glass of ice water with ice cubes floating in it
The glass of water in this picture is filled to the brim.  One more drop, and water would spill over the edge.  When examining the ice you note that the cubes rise just above the surface of the water (like glaciers in the ocean), but do not extend to the bottom of the glass.  Now here’s the challenge: Imagine patiently waiting on a hot summer day until all the ice melts.  What will happen to the water level?  Does it rise and over-flow the glass, remain constant throughout the melting process, or go down?

What do you think will happen to the water level when all the ice has melted?


Think about what’s going on for you as you wrestle with this challenge.  Do you feel like you know the right answer? How confident are you in your response? Are you, like most people who face this challenge, surprised to find that you aren’t sure of the answer, while also feeling conflicted because you think you should know it?  If you answered “Yes” to this last question, then you just experienced the Illusion of Understanding first-hand.

This is a challenging problem for most people – physics students and adults alike. Yet the problem is based on a principle called Archimedes Principle that most of us encountered at some point in a physical science class.

As challenging as the problem is for students, consider how much more daunting it is for a science teacher who wants to help students understand the principle so well that even years later they can confidently use that understanding to solve problems like this one.  I know how daunting the challenge is, because many years ago I was that science teacher.

Here’s the dilemma. As a teacher, I can assure you that it’s very, very difficult to help students develop what I call “authentic understanding” – the kind of understanding that would enable them to answer the iceberg challenge correctly and explain why their answer is correct.  It’s a great deal easier (although not a conscious goal) for a teacher to leave students with the illusion of understanding – the belief that they understand the relevant principle even though they can’t answer questions based on the principle.  I have given the iceberg challenge to hundreds of intelligent, educated adults over many years. Based on their performance I’d say that – despite the best efforts of many capable, dedicated science teachers – authentic understanding in this subject area is relatively rare while the illusion of understanding is quite common.  However if you ask people in the grip of the Illusion whether they understood their science teacher’s lesson on Archimedes’ Principle, many would say “yes” without hesitation.

MOOCs to the rescue?
Let’s carry our self-experiment one step further to see how deep the Illusion goes in this case.  Perhaps Kahn Academy can help you solve the iceberg challenge (assuming schooling has not). Khan’s curriculum on Fluids, Part 5 and Part 6, constitute a formal presentation that, in principle, should allow you to solve the problem as posed above. I invite you to watch those two videos now and try to answer the Iceberg Challenge again.

(Go ahead and watch the videos now.  I'll wait…)

How did you do?  Were you able to resolve the iceberg challenge? Do you feel more or less confident in your answer now?  Khan claims that the ability to control the pacing of the video and the opportunity to re-watch the session will help. You may want to test those assumptions.

How to Develop Authentic Understanding
I have found that Khan – like the many others who use similar instructional strategies both online and off – are overlooking over a hundred years’ worth of discoveries in the learning sciences. Below, I list five major discoveries that define requirements for achieving authentic understanding (see the companion article published in this month’s Journal of Asynchronous Learning Networks for additional detail):
  1. Authentic understanding depends on hierarchically organized knowledge.
  2. Authentic understanding is grounded in direct experience.
  3. Authentic understanding is stabilized by practice (generally at every level within the hierarchy).
  4. Authentic understanding requires formative feedback.
  5. Authentic understanding is context-sensitive.
When these insights are ignored, as they are most of the time in online instruction, educators and students risk reinforcing and perpetuating the Illusion of Understanding, which I have observed in many classrooms and in many countries.  Typically the illusion unfolds in dramatic fashion when teachers ask students to explain their answers, and the students suddenly realize they can’t. Students find themselves speechless or stuck in a rambling explanation that doesn’t even make sense to them.  I have even observed students give a correct explanation and then admit they didn’t understand what they just said.  Now, as the teaching and learning enterprise unfolds on a world stage through a variety of online platforms, we face the risk that the Illusion will be even more widespread and difficult to dispel than ever.

Assuming that your struggle with the above iceberg challenge is no different than almost everyone who attempts the challenge, which of the five observations seem relevant? Did you feel that you lacked the perquisite experiences to reach an answer? Did your past experiences feel relevant yet unconnected or unreliable? Did you feel like you needed to practice some kind of mental exercise but did not know which or how?  These questions all underlie the complicated teaching and learning environment that lead to authentic understanding.  I don’t claim that the process is easy, but the investment is necessary. The challenge here should also underscore another illusion- that is, the illusion that achieving expert status in one discipline - as a hedge fund manager, for example - automatically transfers to another discipline, such as teaching. Teachers, like hedge fund managers, spend decades to become competent at their craft.

If you do watch the videos, which of the five observations seem to be relevant to your experience of understanding?  You will note that Khan does use a similar challenge in parts five and six of his video series, but the context is different.  Does that matter? In Part II we explore in further detail the Illusion of Understanding in the area of math and explore what choices may be available to Khan and all educators, especially those who work online, to better support authentic understanding.

(Continued in Part II which is available here.)

45 comments:

  1. Replies
    1. Thanks, Proofrock.
      (I appreciate the homage to T.S. Eliot, btw).

      We're working on it as we speak!

      Delete
  2. Thank you for writing this - very well done!

    I would recommend this video, from Derek Muller (Veritasium) on a related topic: "Khan Academy and the Effectiveness of Science Videos"

    http://www.youtube.com/watch?v=eVtCO84MDj8&list=PL772556F1EFC4D01C&index=26

    Very germane to this discussion.

    ReplyDelete
    Replies
    1. Thanks for sharing the link, Mike. I will definitely check out this video.

      Delete
    2. I watched the video and you're right - it's *very* relevant to this discussion. Thanks much for pointing us to it!

      Delete
  3. Have you looked at Hubbard's research on the subject of learning, specifically what he called the "Three barriers to study?"
    I've been using that stuff all my life and it is gold!
    http://www.appliedscholastics.org/study-tech/a-breakthrough-in-learning.html

    ReplyDelete
    Replies
    1. This is new to me, Florian. Thanks for sharing - I will take a look.

      Delete
  4. I could be wrong, but in my opinion you fall for an illusion yourself. There is a difference between understanding and skill. One is a variable and the other is a constant. When I have to solve a short task, why should I reduce the liquidity of my brain and imprint this solution as the median of all similar tasks although I do not know the range?
    Same goes for your example about authentic understanding when students cannot give an answer although they should know, have they been taught the range of tasks? Have they been taught the range of solutions? Was there a chance to create a pattern, an emergent knowledge?
    And how much does this affect my life when I have to pay a rent or had a big argument with my love? Emotions easily override and overwrite any common sense. That's why so many child prodigies fail once they reach puberty. Conclusion, if you want authentic understanding of a direct connection between task and solution, then why is science and education so emotionless? And why is this webpage clinically black and white?

    ReplyDelete
    Replies
    1. Hi, Harry. Thanks for joining the conversation.

      > Same goes for your example about authentic understanding when students cannot give an answer although they should know

      Just to be clear: the point made in the blog is not that "students cannot give an answer although they should know [the answer]." The situation is that students *believe* they know the relevant concept but are themselves surprised and dismayed to discover that they *don't* actually know it.

      Delete
  5. Powerful,argument Marc,you have dared where many haven't. I find a lot of sense in your concept of authentic understanding. Your concept is supported by the schema theory: schema activation, where the topic has to be introduced via various strategies, schema construction where new concepts are derived principles are connected, and lastly on refinement, where we expect the student or learner to demonstrate transfer of knowledge is solving a problem. Okey 9 steps of learning cover the learning process along similar lines.

    ReplyDelete
  6. I see at least three topics to discuss in this blog/comments:
    1) Effectiveness of video instruction: Are they effective? In what context? What evidence exits that this model is more (or less) effective than another? Why would an instructor use this tool over another? Why would a student use this tool over another?
    2) Illusion of Understanding: Has this happened to anyone posting (or have you observed this?) How do our current methods of assessment add to this illusion? Do they add to this illusion? What tools exist that allow teachers to assess understanding that is authentic?
    4) Let’s assume that the illusion of understanding permeates our current education system PK-16 (and, btw, I believe it does) in what ways can we provide instruction to instructors to help them recognize this and, more importantly, reverse or remediate instructional models that add to this illusion?
    It is well and good (and necessary!) to discuss these issues, but what we need is action based on discoveries. That begins with first acknowledging this in ourselves; and being mindful of when this happens! Daniel Kahneman provides a stellar analysis of how we fall prey to the illusion of understanding in Thinking: Fast and Slow. This issue isn’t just one related to the world of education. The illusion of understanding affects our political views, our personal choices, our lives…
    What are you doing TODAY to begin to recognize and change your own illusions?

    ReplyDelete
    Replies
    1. These are great questions, Rene. In Part 2 we'll start to wrestle with some of them and we would love to hear your thoughts on that.

      Delete
  7. Oops...yes, I can count...just did not check for spelling/typos...
    Looking forward to further discussion and follow-on article!

    ReplyDelete
  8. What's the answer? You're killing me here. I'm guessing that the buoyancy of ice is inversely proportional to the density compared to water, which is lower, so as it melts, the water will stay at the same height.

    ReplyDelete
    Replies
    1. Hi, Dave.
      I know how you feel. :)

      We don't want to irritate anyone by withholding the answer. But we also believe that giving you the answer would actually do you and anyone reading this a disservice. If we gave you the answer, we'd be robbing you of the opportunity for authentic understanding. That's not something one person can give to another directly like that.

      Delete
  9. I found this an immensely interesting article which really made me question my own teaching methods and how I might redesign materials to encourage the understanding that is preferable. I am very much looking forward to Part II and the further challenges it will present me with as a Lecturer. Thank you .

    ReplyDelete
    Replies
    1. Thanks for the feedback, Angela. This is really wonderful to hear.

      Delete
  10. All good discussions and worth exploring but to me there are two illusions that contradict a personal philosophy about education and learning. the Khan approach makes assumptions about student access to their resources and ignores some basic social justice principals associated with many school districts. many students still cannon access the internet at home..its a cost issue and secondly many have to act as carers as both parents are at work. A truly authentic flip should be totally during school time for it to be equitable. I see many examples in the US that are contrived and do not address low socio economic schools needs. The second illusion is that Khan can substitute for real hands on learning that focuses upon the lifeworlds of a particular cohort of students.

    Enough for now...Your thoughts!

    ReplyDelete
  11. Online learning?

    I don't think that "watching videos" is the same as online learning!

    We all agree that learning needs context and many tools of online learning can carry a fair part of the context. Think of LMS, forums, or even portfolios.

    I like the impression "false understanding"... but that is something that also happens if you learn math recipes by hand. The criteria listed here make sure, for example, that there are varied exercises to be performed as part of a normal math or physics learning. Makes sense, is commonly applied.

    ReplyDelete
    Replies
    1. Hi, polx.

      > I don't think that "watching videos" is the same as online learning!

      You raise an interesting question. Currently, people use the term "online learning" to refer to a wide range of activities. The only thing they seem to have in common is that they require internet access. Do you have thoughts on how we might make a more precise definition of the term?

      Delete
  12. Michael,

    I wonder if this should be much constrained.
    Maybe... A learning process with the usage of a computer accessing the internet?

    paul

    ReplyDelete
  13. There is a trick in this question. Water behaves differently when it's ice, so this was confusing. I still don't know if this is relevant.

    Apologies, but I have read the "authentic understanding" concept before, and I do not see how it can be operationally defined. "Accurate understanding" might be more descriptive. The opposite is the "illusion of understanding (Robert Bjork 1996 or so) or "illusion of knowledge", Glenberg, 1982.)

    ReplyDelete
    Replies
    1. Hi, haa.

      There's no trick in the question. It's intended to be a straightforward question about physics.

      Thanks for these references to Bjork and Glenberg. I wasn't aware of their work but will look it up. I did find this blog post that discusses "Illusions of Understanding and Knowledge" after searching for Bjork based on your suggestion:
      http://www.edge.org/response-detail/23731

      It looks interesting and I will follow up more, so thanks for sharing those.

      As for operationally defining authentic understanding, Part 2 (which should be available later today) will address that very question. I hope you will let us know what you think after you read it!

      Delete
  14. Part 2 of the blog post is up now:
    http://theeducationscientist.blogspot.com/2014/03/khan-academy-how-does-it-measure-up.html

    ReplyDelete
  15. If you really desire to get such type of information, visit this blog quickly.Are you needing an outline for you for your essay?

    ReplyDelete
  16. Much gratitude to all of you that much for such better than average and incredible article on this at this time.Really your article is having particularly useful data which can help other individuals somehow. Regards Sniper Fury APK

    ReplyDelete
  17. I have read your blog its very attractive and impressive. I like it your blog.

    .Net Online Training

    Dot Net Online Training | ASP.NET Online Training

    ReplyDelete
  18. JavaScript is the language future. The new JavaScript Version (Ecmascript 6) has been released.

    Javascript Online Training | Angularjs Online Training


    Java Online Training

    ReplyDelete
  19. Java Online Training Java Online Training Java Online Training Java Online Training Java Online Training

    Hibernate Online Training Hibernate Online Training

    ReplyDelete
  20. This article aims to post a practical introduction as to how doctorate dissertations should be viewed and should be written. This way, doctorate level Top Dissertation Writing Service UK writers would have a better sense of understanding on how important these particular kinds of writing projects are and how serious doctorate level learners should take the challenge of completing their dissertations successfully.

    ReplyDelete
  21. Pleasant Tips..Thanks for Sharing….We keep up hands on approach at work and in the workplace, keeping our business pragmatic, which recommends we can help you with your tree clearing and pruning in an invaluable and fit way.
    python Training institute in Pune
    python Training institute in Chennai
    python Training institute in Bangalore

    ReplyDelete
  22. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.

    rpa training in bangalore
    best rpa training in bangalore
    RPA training in bangalore
    rpa courses in bangalore

    ReplyDelete
  23. Great Article… I love to read your articles because your writing style is too good, its is very very helpful for all of us and I never get bored while reading your article because, they are becomes a more and more interesting from the starting lines until the end.
    Data Science course in Chennai
    Data science course in bangalore
    Data science course in pune
    Data science online course
    Data Science Interview questions and answers
    Data Science Tutorial

    ReplyDelete
  24. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.Best Devops Training in pune
    Microsoft azure training in Bangalore
    Power bi training in Chennai

    ReplyDelete
  25. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    rpa training in bangalore
    best rpa training in bangalore
    rpa training in pune | rpa course in bangalore
    rpa training in chennai

    ReplyDelete
  26. Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article. thank you for sharing such a great blog with us.
    rpa training in bangalore
    best rpa training in bangalore
    rpa training in pune | rpa course in bangalore
    rpa training in chennai

    ReplyDelete
  27. Does your blog have a contact page? I’m having problems locating it but, I’d like to shoot you an email. I’ve got some recommendations for your blog you might be interested in hearing.
    AWS Training in Chennai |AWS Training Course in Chennai
    AWS Training in Chennai | AWS Training in Chennai cost
    AWS Training in Chennai | Aws Training Institute in Chennai

    ReplyDelete
  28. Thank you for allowing me to read it, welcome to the next in a recent article. And thanks for sharing the nice article, keep posting or updating news article.
    AWS Training in Bangalore

    ReplyDelete
  29. Great blog thanks for the author who wrote this article
    power BI training in chennai

    ReplyDelete

  30. very informative blog and useful article thank you for sharing with us , keep posting learn more Technology

    Tableau online Training

    Android Training

    Data Science Course

    Dot net Course

    iOS development course

    ReplyDelete