"Exam wrappers are short activities that direct students to review their performance (and the instructor's feedback) on an exam with an eye toward adapting their future learning (again, see Appendices A1 and A2 for two samples).  Exam wrappers ask students three kinds of questions: How did they prepare for the exam?  What kinds of errors did they make on the exam?  What could they do differently next time?"

Tomorrow's Professor Msg.#1340 Design Considerations for Exam Wrappers

 

Folks:

The posting below looks at the use of exam wrappers as a way to help students improve their performance on exams. It is from Chapter 2 - Make Exams Worth More Than the Grade: Using Exam Wrappers to Promote Metacognition, by Marsha C. Lovett, in the book,  Using Reflection and Metacognition to Improve Student Learning Across the Disciplines, Across the Academy, edited by Matthew Kaplan, Naomi Silver, Danielle LaVaque-Manty, and Deborah Meizlish. Published by Stylus Publishing, LLC. 22883 Quicksilver Drive, Sterling, Virginia 20166-2102. [http://www.styluspub.com/Books/Features.aspx] Copyright © 2013 by Stylus Publishing, LLC. All rights reserved. Reprinted with permission.

Regards,

Rick Reis
reis@stanford.edu
UP NEXT: Putting E-Portfolios at the Center of Our Learning

Tomorrow's Teaching and Learning

---------- 2,263 words ----------

Design Considerations for Exam Wrappers

To devise a practical and effective intervention that helps students develop metacognitive skills, we need to consider several realities of teaching and learning.  First, however much instructors may value metacognition, they still face great pressure to cover the course content (especially in introductory courses) and hence must allocate their class time sparingly.  So any practical intervention must impinge minimally on class time.  Second, today's college students are busy and as such tend to be highly sensitive to the time they spend on course-related activities, especially if they do not see a connection to their grade or the course content.  So students must be able to complete any practical intervention within the (likely small) amount of time they are willing to invest.  Third, courses vary widely in many different ways, including the disciplinary content (e.g., biology versus physics), format of the course (e.g., lecture versus small-group discussion), and types of activities (e.g., problem set versus essay), and instructors do not want to have to design a distinct instrument for every course.  So any practical intervention must be easily adaptable across diverse course features.  Fourth, in order to produce significant learning gains, instructors need to give students repeated practice opportunities.  Moreover, the repetitions need to allow enough variety, to avoid seeming dull and predictable, and to support transfer.  So any effective intervention must be repeatable and yet flexible enough to accommodate variation in format.  Finally, and most importantly, metacognition will not improve unless students are actively engaging in metacognitive practice of the sort discussed earlier.  In other words, to be most effective, the intervention must be (a) targeted on the metacognitive skills that instructors want their students to learn, (b) repeatable for multiple practice opportunities, (c) exemplified in diverse contexts, and (d) grounded in the content of students' disciplinary learning.

For the reasons identified in the beginning of this chapter, exams offer an ideal platform for achieving metacognitive gains. My approach was to build metacognitive practice around exams and in so doing satisfy the previously listed constraints.  As the name suggests, this is what exam wrappers are all about.

What Are Exam Wrappers?

Exam wrappers are short activities that direct students to review their performance (and the instructor's feedback) on an exam with an eye toward adapting their future learning (again, see Appendices A1 and A2 for two samples).  Exam wrappers ask students three kinds of questions: How did they prepare for the exam?  What kinds of errors did they make on the exam?  What could they do differently next time?  Each of the question types is discussed next.

1. How students prepared for the exam.  Asking students to reflect on how they prepared for the exam forces them to confront the choices, explicit or implicit, they made about their studying.  This prompts students to consider issues such as whether they studied enough or sufficiently in advance.  Similarly, asking students which of various study strategies they employed (e.g., reviewing notes, solving practice problems, rereading the textbook) highlights that there are many options they could have taken.  It also presents various study strategies that students might not have even considered and thus suggests some new possibilities for how they might prepare differently next time.

2. What kinds of errors students made.  Once they have received a grade, students do not always think carefully about their performance on an exam.  If they did well, they might mark it as a success without much further thought; if they did poorly, there's a strong temptation to leave the painful event behind.  Thus, the second set of questions posed in exam wrappers is designed to encourage students to analyze their performance in greater depth, giving students something constructive to do with the feedback a graded exam offers.  One way to do this is to identify the critical components or stages of the tasks on the exam and have students estimate their degree of difficulty (e.g. how many points they lost) with each component.  For example, did they read the question carefully, did they have trouble "setting up" the problem, did they fail to understand the concepts involved, or did they make mistakes on the required arithmetic or algebra?  Focusing students' reflection at this level informs their analysis of their own performance.  Moreover, the labels for the different possibilities provide a concrete language for students to use when assessing their own performance.  Note that this part of the exam wrapper is a natural place for instructors to tailor the questions to their own needs.  For example, the exam wrappers in Appendices A1 and A2 use different labels to fit the needs of two disciplines: physics and calculus.  Instructors may also want to adapt these labels to include specific misconceptions or difficulties that have been revealed in their course by past students.  Or, they may want to include more general issues that affect students' exam performance.  For instance, an instructor I worked with recently was concerned about test anxiety adversely affecting her students' exam performance, so she incorporated this issue into her exam wrapper.

3. How students should study for the next exam. Students can use their responses to the first and second types of exam-wrapper questions to think about how they should approach the next exam.  A key goal of the third type of exam-wrapper question is to help students see the association between their study choices and their exam performance so they can better predict what study strategies will be effective in the future.  One way to do this is to ask students to look back at their responses to the first two parts of the wrapper and then to list specific ways they might prepare differently for the next exam to improve their performance.  Another option is to prompt students to attribute their various difficulties (from part two of the wrapper) to specific study strategies (from part one) they did or did not employ.  Rather than merely telling students to "study harder" or "do more practice problems before the exam," this third type of exam-wrapper question helps students discover effective study strategies on their own.  In effect, exam wrappers are asking students to give their future selves advice.

Why Exam Wrappers Work

The earlier section on design considerations described five constraints that a metacognitive intervention should satisfy to be practical and effective.  Understanding the ways in which exam wrappers satisfy these constraints helps explain why they work.

1. Impinge minimally on class time.  Exam wrappers require only a few minutes and are completed at a time when students arguably are somewhat distracted anyway.  Moreover, instructors can eliminate even the minimal impact on class time by giving exam wrappers as part of homework.

2. Be easily completed by students within the time they are willing to invest.  With typically only one page of questions, none of which requires much writing, exam wrappers take relatively little student time.

3. Be easily adaptable.  As we have seen, exam wrappers include three main question types, and these question types can be applied to almost any course - as long as it has exams.  (See the following section, How to Use Exam Wrappers, for different ways of implementing exam wrappers.) In addition, this general approach can be applied to any kind of graded assignment.  (See the section titled Other Kinds of Wrappers for a brief discussion of other ways to employ this type of tool.)

4. Be repeatable yet flexible.  The core questions being asked in an exam wrapper do not diminish in value when asked repeatedly.  At the same time, it is easy to adjust the details of exam wrappers so as to keep the exercise fresh.  Instructors can easily vary the specific content of exam wrapper questions, add new questions, and tailor the questions to their particular instructional situation. (See Appendix B for two additional exam wrappers beyond those presented in Appendices A1 and A2.)

5. Exercise the skills instructors want their students to learn.  The reflection required to complete an exam wrapper leads students to assess their own strengths and weaknesses, evaluate their performance, identify which strategies work for them, and generate appropriate adjustments.  These are key metacognitive skills that many instructors want to promote.

How to Use Exam Wrappers

Here I describe a basic recipe for how to use exam wrappers, along with variations and options instructors may find useful.

Step 1: Students prepare for and take the first exam using their typical study strategies.  No special intervention is needed for this first exam.

Step 2: The instructor gives students the exam wrapper instrument when the graded exams are returned and asks students to complete the exam wrapper as soon as possible upon seeing their exam performance.  Ideally, this is done right then in class and need only take 10 minutes of class time.  But there are other possibilities.  Students can complete the exam wrappers as homework, submitting their responses by a specified deadline.  Or students can complete the exam wrappers online as a nongraded assignment (e.g., within a course management system or online instructional environment or with an online survey system).

Step 3: The instructor collects the exam wrappers.  Although exam wrappers are not graded activities, it is important to collect them because (a) the exam wrappers will need to be returned to students at a later point (see Step 4) and this prevents them from getting lost, and (b) the instructional team may want to review students' responses to gain insight into their students' learning that they otherwise might not be able to obtain.  In particular, the instructor or teaching assistants can skim students' responses to see whether there are patterns in how students analyzed their strengths and weaknesses or in how students described their approach to studying for the exam.  For example, instructors may be surprised to learn the amount of time students spent studying (either how much or how little), when students chose to start their studying (e.g., 2 a.m.. the night before), or how students spent their study time (e.g., memorizing formulae rather than solving practice problems).  The instructional team can also consider the additional instructional support students said they would like to receive and possibly provide something along these lines.  A wide variety of adjustments may be suggested based on what the instructional team learns from the exam wrappers about students' strengths, weaknesses, and study strategies.

Step 4:  At the time when students should begin studying for the next exam, the instructor returns the completed exam wrappers (from the previous exam) to students.  The idea here is that students review their own recommendations for how to study more effectively, given their own past experiences, strengths, and weaknesses.  Depending on the class format and the time available, there are many variations on this step that instructors can use.  For example, Figure 2.1 shows a set of follow-up questions that can accompany the completed exam wrapper sheet.  These questions prompt students to review their exam wrapper responses and recommit to implement their own suggestions.  Another option (not mutually exclusive) that works well with smaller classes is to give students a few minutes to reread their exam wrappers and then take a few minutes for students to share effective study strategies.  Regardless of which approach instructors take for this step, the key aspect involves reminding students of their own advice and encouraging them to take it.

Step 5 (optional, but desirable): Repeat steps 2 through 4 for subsequent exams.

When an instructor provides exam wrappers regularly across multiple exams, students get repeated practice in applying the skills of self-regulated learning.  This helps students build a habit of mind to monitor their own learning, reflect on their study strategies, and make appropriate adjustments.  For reasons mentioned earlier, it can be useful to include nontrivial variation in the structure of subsequent wrappers while still prompting the desired metacognitive processes.  For example, an exam wrapper used for exams later in the semester can be streamlined compared to the first exam wrapper.  Having the wrapper still gets students to engage in reflection and analysis, but fading the scaffolding encourages students to take on more of the responsibility for the process.

These five steps are easy to implement, take relatively little time, and are very flexible.  The metacognitive practice from using wrappers in a course offers substantial benefits.  And when multiple instructors do this across different courses, students can learn metacognitive skills in multiple contexts, thereby increasing their likelihood of transferring the skills to new learning situations in the future.  This is exactly what we did at Carnegie Mellon University, implementing wrappers in several introductory math and science courses, as described in the next section.

--------------------------------------------------------------------------------------------------
Figure 2.1. Exam wrapper add-on: Additional questions for when completed wrapper is returned.

Physics Pre-Exam Reflection                                     Name: ______________

You will soon be taking Exam #2.  This sheet poses a few questions to help you reflect on your experience with Exam #1 so that you can prepare effectively for Exam #2.

1. Read through your responses on the Post-Exam Reflection sheet from the last exam.  (Your TAs will hand back your sheet.)  Jot down anything your read that you think will be helpful for you as you study for the next exam.

2. Self-assessment involves analyzing your own strengths and weaknesses.  This can be helpful in deciding what you should study more or less.  Given your responses on the Exam #1 reflection sheet along with your experiences learning new topics for Exam #2, what do you think you should study most as you prepare for Exam #2?

3. Read your response to question #4 on your Post-Exam Reflection sheet.  Write down how you plan to implement your own suggestions to study effectively for Exam #2.

 

 

 

 

----------------------------------------------------------------------------------------------------
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING
----------------------------------------------------------------------------------------------------