The posting below looks at very interesting diagnostic learning approach that using a varying difficulty - varying weight testing format. . It is by Randy M. Isaacson Indiana University South Bend Christopher A. Was Kent State University and is #53 in a series of selected excerpts from the NT&LF newsletter reproduced here as part of our "Shared Mission Partnership." NT&LF has a wealth of information on all aspects of teaching and learning. If you are not already a subscriber, you can check it out at [http://www.ntlf.com/] The on-line edition of the Forum--like the printed version - offers subscribers insight from colleagues eager to share new ways of helping students reach the highest levels of learning. National Teaching and Learning Forum Newsletter, Vol. 19, Number 5, September 2010. 2010.© Copyright 1996-2010. Published by James Rhem & Associates, Inc. All rights reserved worldwide. Reprinted with permission.
UP NEXT:During 'Dual Career' Right
------------------------------------------ 1,471 words ----------------------------------------
Building a Metacognitive Curriculum: An Educational Psychology to Teach Metacognition
Most college students are very likely to respond primarily to the resources in a class that will impact their grade. As much as educators would like to believe their students are intrinsically motivated, thirty years of teaching has dissuaded us from expecting that our students will enter class with an insatiable interest in educational psychology. This has led us to accept their performance orientation and build a curriculum with many resources to improve their grades. Early in the semester many students fall far short of their goals, and so we teach them self-regulated learning (SRL) skills based on careful reflection about why they are failing and what strategies they need to develop to improve their grades. It is important to help students realize that the first step in changing their learning is to improve their metacognitive skills- to become an effective learner you need to know-what-you-know.
Monitoring Vital Signs
By creating a learning environment with extensive immediate weekly feedback that reveals the specific deficiencies in their metacognition and learning strategies, students are more likely to reflect on how they are learning and what they have successfully learned. We have developed an extensive assortment of learning resources, which focus on knowing-when-you- know. The first step in helping students to improve their self- regulation and metacognition was the development of a test format that directs them to reflect on how they study and how accurate and confident they are in their understanding of the concepts and each question on the test.
Variable Difficulty - Variable Weight Tests
Students are given weekly tests that require them to make meta- cognitive choices that will improve their test score. The basic mechanism of the Variable Difficulty - Variable Weight testing format is the presentation of three levels of test questions, each of which theoretically requires a different level of learning, and then allowing students to choose which questions they wish to answer. Two sections of the test (Level I and Level II questions) have fifteen questions- students choose ten about which they are confident (indicated by placing their response on the left side of the answer sheet) and five questions about which they are less confident. One section of the test (Level III questions) has five questions-students choose three about which they are confident and two about which they are less confident (placed on the right of the answer sheet). Each section of the test is weighted differently and the students' confidence rating impacts their test scores.
Level I questions are basic objective test questions that require only knowledge and comprehension. Since these questions are very basic and can be answered correctly using simple rehearsal strategies, they are worth fewer points. Level II questions are also objective questions, but they require the application of psychological concepts to classroom settings. The typical question describes a classroom problem and asks the students to solve the problem using concepts from class. These questions are typically more difficult and require elaboration.
Level III questions are also objective questions, but they are much more difficult and are designed to measure the student's understanding of the structure of the material being learned. There are a number of formats for Level III questions but the most common are analogy and hierarchy questions. Students are encouraged to study for these questions by organizing items of related content into structures that demonstrate relationships.
Self-Knowledge and Hedged Bets
For each of the levels, the questions for which the students indicate they are confident of their answer and in fact answer correctly are weighted more heavily than questions they answer correctly but indicate less confidence. Also, as difficulty increases so does the weight of correct responses for confident responses, but not for unconfident responses.
Students quickly learn that making accurate metacognitive choices dramatically improves their test scores; students can receive 88% of the test points for getting 66% of the questions correct and the left-right answer sheet makes their metacognitive mistakes very obvious. After each weekly test, students are asked to reflect on their test and record their left-right points and scores in a journal. The left-right test format is also used with a weekly quiz. Of the many resources used in this course, the left-right test format is likely to have the most powerful impact on student metacognitive knowledge monitoring because of the grade incentive.
"Clickers" and Metacognition
At the start of every class, students are asked a "Question of the Day" that requires them to indicate whether they are: Absolutely Sure, Fairly Sure, or Just Guessing at their answer. The points for their response are dependent on the correctness of their answer and the confidence they state about their answer. For example, if students indicate they are absolutely sure, they earn nine points if they are correct, but no points if they are wrong. However, if they indicate they are unsure or just guessing, they earn three points if they are correct and two points if they are wrong. Students, again, quickly learn that carefully reflecting on their confidence improves their grade.
To point out the metacognitive errors made by many students, every student has a partner. After each partner answers the Question of the Day, the pairs are given two minutes to discuss the question and then record a second answer for fewer points. One of the most damaging dispositions to the relationship between metacognition and learning is what is known in signal detection theory as a "false alarm": This occurs when students have a feeling of knowing that is overconfident when in reality they do not know. A student response system that includes confidence ratings allows students to give daily feedback to their partner about their reasoning and their confidence.
If students use all the available resources, they can make up to 1,610 metacognitive choices (Are you Absolutely Sure, Fairly Sure, or are you Just Guessing?) during the semester.
Students have access to many online resources which emphasize metacognition. For example, to prepare for the weekly tests, students have the option of using an online metacognitive practice test, which presents them with 15 practice test questions (in a variable difficulty - variable weight format).
Each student is enrolled in a Peer Mentor Learning Commu- nity of approximately six to eight students led by two peer mentors. These discussion groups have a variable difficulty - variable weight quiz each week that also has a left- right format. The peer mentors direct the students to think-about- their-thinking and encourage each student to complete and submit (for points) a Peer Mentor Learn- ing Community Journal each week. The Journal is a reflection on the student's use of self- regulated learning and metacog- nition. For example, students keep track of the accuracy of their left-right choices on the weekly test and quiz to help focus their atten- tion on metacognition.
Students are also required to complete weekly self-reflections which focus on self-regulated learning and metacognition. Early in the semester, students are asked to reflect on the impact of knowing-when-you-know on their test scores (left-right test format) and their clicker points. Mid-semester the self-reflection asks students to examine how their metacognition has changed during the semester. At mid-semester students are invited to write a self-assessment paper which focuses on self- regulated learning and metacog- nition. The self-assessment re- quires an in-depth examination of how they regulate their behavior, metacognition, motivation, and their cognitive study strategies, including making a plan to improve their learning during the last half of the semester.
What We Have Learned
Research points to a significant relationship between metacognition and learning, and our own research indicates that improvement in metacognition is related to the techniques described above (e.g., Isaacson & Was, 2010). One of the challenges to this area of the scholarship of teaching and learning is measuring meta- cognitive knowledge monitoring in the classroom. The Knowledge Monitoring Assessment (see Tobias and Everson, 2009, for a review) is an effective tool for measuring metacognition with a broad stroke. The left-right test format described begins to explore the metacognitive choices students make, while also giving them an incentive to make accu- rate choices and reflect on their improvement.
It is important to note that metacognitive classroom techniques are not limited to objectivist courses or courses that rely on objective tests. For example, Concepción (2004) describes how he uses metacognitive techniques to increase his philosophy students' comprehension while reading philosophy. Simply having students answer self-assessment questions while reading and compare self-reflections of understanding with those of other students has increased their understanding of philosophical arguments. Kelleher (1997) describes how she uses Readers' Theatre to teach students meta- cognitive strategies.
The clear link between meta- cognition and learning warrants that a focus on metacognition be an integral part of the curriculum. Metacognition is a skill that can and must be developed in the context of the learning environment.
Concepción, D. W. 2004. "Reading Philosophy with Background Knowledge and Metacognition." T eaching Philosophy 27(4): 351-368. Isaacson, R., & Was, C. 2010, May. "Believing You're Correct vs. Knowing You're Correct: A Significant Difference?" Paper presented at the annual conference of the American Education Research Association, Denver, CO. Kelleher, M. E. 1997. "Readers' Theatre and Metacognition." New England Reading Association Journal 33(2): 4-12. Tobias, S., & Everson, H. 2009. "The Importance of Knowing What You Know: A Knowledge Monitoring Framework for Studying Metacognition in Education." Pages 107-128 in Handbook of Metacognition in Education (ed. D. J. Hacker, J. Dunlosky, & A. C. Graesser; New York, NY: Routledge).
Randy M. Isaacson Indiana University South Bend South Bend, IN 46615
Christopher A. Was Kent State University Kent, OH 44242
Phone: 330-672-2929 Fax: 330-672-2512 E-mail: firstname.lastname@example.org