"Empirical evidence confirms that the combination of high expectations and low stakes (exactly the conditions of a good video game) matter for learning."

Tomorrow's Professor Msg.#1271 Designing College More Like a Video Game

 

Folks: 

The posting below looks at how to use the video game model to provide assessments that promote learning by combining low-stakes and high-quality feedback.  It is from Chapter 4, Designing College More Like a Video Game, in the book, Teaching Naked: How Moving Technology Out of Your College Classroom Will Improve Student Learning, by José Antonio Bowen. Published by Jossey-Bass, A Wiley Imprint. 989 Market Street, San Francisco, CA 94103-1741-www.josseybass.com. Copyright © 2012 by John Wiley & Sons, Inc. All rights reserved. Reprinted with permission.
 
Regards,

Rick Reis
reis@stanford.edu
UP NEXT: To MOOC or Not?
                                                                                                 Tomorrow's Teaching and Learning

                                                                           ----------------------------------- 1,672 words ----------------------------------

                                   Designing College More Like a Video Game - Motivating Change with High Standards and Low Stakes
We can open doors, but students have to walk through them. How then do we motivate students who have been highly successful in high school (using one part of their brain and one way of thinking) to abandon that mode of thinking and learn to think in new ways using new parts of their brain that are not yet fully developed? We are asking students to discard the very patterns of thought that have led them to success. We are asking for substantial change under conditions (awarding grades) that punish failure and at a time (the beginning of college) when anxiety about change and failure is at its fullest.

Pintrich, Marx, and Boyle (1993) demonstrate that motivation is an important factor in leading students to conceptual change. After failure it is natural to ask what went wrong, but few of us do so after success. Thus, it is easier to teach change after failure than after success, so we need more planned failure in the college experience.

Thus, the traditional passive method of college teaching (I.e., lecturing) is less effective than active learning in developing higher-order cognitive skills. Delivering content alone has virtually no effect on students' beliefs about the world. Students can memorize data that conflict with their beliefs, but without active engagement with the new material, in the form of discussions, writing, debates, projects, and hands-on application, they do not really confront the implications of the new content.

For example, Mazur (1996) discovered that his Harvard physics students were very good at memorizing but not at learning to understand. Students liked him, did well on his tests, and were able to recite formulas but left his courses with little ability to solve problems, virtually no conceptual knowledge, and certainly no love of physics. Improving the quality of his lectures did not improve student learning, so he created a new method of peer instruction that begins with giving students a reading before class. In class he asks students to think about a question, then to answer, and then to convince a neighbor of the right answer. He then asks students to answer the question again and provides an explanation before starting the process over with a new question. He found that even when he never worked a problem in class, he could improve students' ability to solve problems by allowing them to practice on each other. Embedded in Mazur's methods are high standards-conceptual understanding instead of mere memorization-but low stakes, in the form of no penalty for wrong answers in class.

Empirical evidence confirms that the combination of high expectations and low stakes (exactly the conditions of a good video game) matter for learning. Arum and Roksa (2011) even quantify how high the standards need to be: demanding faculty require over 40 pages a week of reading and more than 20 pages of writing per semester (p.93). Even at highly selective American colleges, they found that only 68% of students experienced both standards in any one semester (p. 73). They also discovered, however, that the best faculty not only expected and demanded more but also were approachable (p. 93). There are structural ways to reduce the stakes (e.g., have more exams worth fewer points or allow students to retake exams), but being approachable and supportive also improve learning.

Bain (2004) came to the same conclusion: The best teachers focus on challenging students in a supportive environment where failure is tolerated. The combination is essential; just having high standards is not enough to help students learn. Bain discovers repeatedly that the best teachers expect more of their students yet treat them with genuine caring and give them a sense of control. Students learn best when they believe that the professor wants them to succeed.

We know that the brain is most easily changed when different parts are engaged and students feel they are making progress. Students have to be motivated and feel safe. Understanding and reconciling these two seemingly disparate concepts of high standards and low stakes is critical for successful course design: Bain (2004) concludes that we need to combine "faith in abilities, concentration on outcomes, rejection of power in favor of creating opportunities, and the perception that external factors do make a difference" (p. 83). The personal and the intellectual are intertwined

Practically, our ability to lower the risk of failure while maintaining high standards means we have to rethink what and how we assess. Walvoord and Anderson (1998) point out that grading and assessment not only are about evaluation but also are an important part of the environment and motivation for learning. We can reduce anxiety and increase the opportunities for change by combining clear learning outcomes with lots of low-stakes assessment. Lowering stakes can consist of more exams worth fewer points each, more chances to practice skills and experiment with concepts, more drafts, second and third chances for resubmission and regarding of assignments, feedback on drafts, practice tests, less time pressure, discussion that rewards risk, reduced competition for grades (e.g., pass-fail grading or grading against a fixed criterion instead of using a curve, which increases competition), and an environment of shared interest and exploration. It is precisely the combination of challenging engagement and low consequences for failure that has proved so potent in the video game industry: if games do not provide both enough pleasant frustration and positive feedback, they do not succeed in the marketplace.

Good assessments also need to motivate students, but motivation is complex, variable, situational, and personal. Motivation varies with our interest but also with our expectations of success, effort, and intrinsic value of the task (Pintrich, 1988, 1989; Wolf, Smith, & Birnbaum, 1995). As a group, faculty are highly motivated by intrinsic value of a task (Froh, Menges, & Walker, 1993) and feelings of competency (Blackburn, Lawrence, Bieber, & Trautvetter, 1991), but it may also be useful to remind ourselves of the importance to our students of other types of motivation. High expectations can be motivating; if someone else believes we can do this, then perhaps we can. The relationship between motivation and lower-stakes assessment, however, is more complex.

Wise and DeMars (2005) determined that, on average, motivated students outperformed unmotivated students by one-half a standard deviation. Raising stakes can actually be a motivational strategy; increasing the consequences-the strategy of the U.S. Internal Revenue Service-raises test motivation and test performance. Grades, therefore, motivate some students. However, raising stakes also increases anxiety, which can offset any motivational gains (Smith & Smith, 2002). Importantly, students in these studies felt that they could exercise some control over motivation but not anxiety: I can decide to do better, but I can't decide not to be anxious. Low-stakes assessment can reduce anxiety, and that will clearly help some students: about a third of students in Wolf and Smith's (1995) experiment did better on the test that did not count toward the grade. Others in education (Bain, 2004), psychology (Weiner, 1990), and neuroscience (Goswami, 2008) all find that stress inhibits learning. If we can lower anxiety and increase motivation at the same time, we will create optimal conditions for learning.

There are ways to increase motivation with encouragement, rewards, or recognition that do not also raise anxiety. Financial rewards, although impractical for schools, work for most parents. Having an honor roll or dean's list is another way to increase recognition and importance.

Assessments that promote learning combine low-stakes and high-quality feedback. Both foster change and are highly motivating; it is easier to try something new if the stakes are low and easier to change when you are being encouraged and when you know exactly what change is needed. Fink (2003) defines high-quality feedback as being frequent, immediate, discriminating, and loving (FIDeLity).

Creating time for quality feedback takes time and needs to be part of course design. Walvoord and Anderson (1998) suggest separating commenting from grading: offer comments but no grade for an assignment, and then grade the final product without commenting, since at that point most students will ignore the comments (p. 120).

Since effort is inversely related to motivation, making tests and assignments easier and less work improves motivation. Essay questions are perceived as more mentally taxing, so they produce lower motivation compared with multiple-choice questions. Wolf et al. (1995) found that less-motivated students worked harder on questions that seemed less mentally taxing, but this strategy, of course, also lowers standards. Spreading out the same material over multiple tests, however, maintains high standards while lowering the consequences of any individual failure, thus increasing motivation for each assessment. If we want to stimulate and evaluate the critical thinking skills of students, we need ways for students to try this dangerous and life-changing practice in a safe space.

Video games are really just a series of tests, but unlike most college tests they are designed to be intrinsically motivating. Players and students alike are more motivated by tests that are pleasantly frustrating (Gee, 2004) or moderately challenging (Pintrich & Schunk, 2002). Like game designers, we can also create tests that follow a narrative or tackle a problem: in other words, make tests more fun by giving them a story line or having them relate to a problem that will motivate your students. While we tend to focus on creating tests that are fair and reliable, we need also to construct assessments that are focused, moderately challenging, and intrinsically interesting. By giving consideration to the format of exams and the examples we use, faculty can increase motivation and lower stress.

As described in the following chapters, technology creates more opportunities for low-stakes tasks that double as motivators and assessment; we can literally turn courses into video games. The fundamental conclusion from this research, however, is that high standards, low-stakes assessment, and motivation all need to be integrated into course design. Constant low-stakes assessment provides opportunities for practice, risk, and even failure but needs to be paired with ambitious and clear learning outcomes in the context of a broader course strategy and tasks that motivate students to persist.
 

Bibliography

Bain, K. (2004). What the best college teachers do. Cambridge, MA: Harvard University Press

Blackburn, R. T., Lawrence, J. H., Bieber, J. P., & Trautvetter, L. (1991). Faculty at work: Focus on teaching. Research in Higher Education, (32)4, 363-383.

Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass.

Froh, R. C., Menges, R. J., & Walker, C. J. (1993). Revitalizing faculty work through intrinsic rewards. In R. M. Diamond and B. E. Adams (Eds.), Recognizing faculty work: Reward systems for the year 2000. San Francisco: Jossey-Bass, 87-96.

Gee, J. P. (2004). Situated language and learning: A critique of traditional schooling. London: Routledge.

Goswami, U. (2008). Neuroscience and education. Jossey-Bass reader on the brain and learning. San Francisco: Jossey-Bass.

Mazur, E. (1996). Peer instruction: A user's manual. Upper Saddle River, NJ: Prentice Hall.

Pintrich, P. R. (1988). A process-oriented view of student motivation and cognition (pp. 55-70). In J. S. Stark and R. Mets (Eds.), Improving teaching and learning through research. San Francisco: Jossey-Bass

Pintrich. P. R. (1989). The dynamic interplay of student motivation and cognition in the college classroom (pp. 117-160). In C. Ames and M. Maehr (Eds.), Advances in achievement and motivation, Vol. 6. Greenwich, CT: JAI Press.

Pintrich, P. R., Marx, R. W., & Boyle, R. A. (1993). Beyond cold conceptual change: The role of motivational beliefs and classroom contextual factors in the process of conceptual change. Review of Educational Research, (63)2, 167-199.

Pintrich, P. R., & Schunk, D. H. (2002). Motivaiton in education: Theory, research, and aplications (2nd ed.). UpperSaddle River, NJ: Merrill Prentice-Hall.

Smith, L. F., & Smith, J. K. (2002). Relation of test-specific motivation and anxiety for test performance. Psychological Reports, (91)3, 1011.

Walvoord, B. E., & Anderson, V. J. (1998). Effective grading. San Francisco: Jossey-Bass.

Weiner, B. (1990). History of motivational research in education. Journal of Educational Psychology, 82, 616-622.

Wise, S. L., & DeMars, C. E. (2005). Low examinee effort in low-stakes assessment: Problems and potential solutions. Educational Assessment, 10, 1-17.

Wolf, L. F., Smith, J. K., & Birnbaum, M. E. (1995). Consequence of performance, test motivation, and mentally taxing items. Applied Measurement in Education, 8, 341-351.



* * * * * * * 
NOTE: Anyone can SUBSCRIBE or UNSUBSCRIBE to the Tomorrows-Professor Mailing List by going to: 
https://mailman.stanford.edu/mailman/listinfo/tomorrows-professor

----------------------------------------------------------------------------------------------------
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING
----------------------------------------------------------------------------------------------------