"I reluctantly decided that I should give each exam twice. Initially, so as to not to cut into coverage, I offered the second try in the evening at a time possible for everyone who wanted to take the exam. Students kept the better of the two grades. Performance improved markedly. I ultimately saw that studying twice for exams (which not every student did) taught on average more content than another lecture would have. I then started giving both exams in class time. Once this approach to exams proved successful, I adopted it in all courses (Nelson, 1996, in press-a)."

Tomorrow's Professor Msg.#1059 Dysfunctional Illusions of Rigor: Part 2 - Advanced Illusions

 

Folks:

The posting below looks at illusions of rigor we often accept in teaching our courses and what in fact are more realistic approaches that can provide the same high quality outcomes. It is intended to as a follow-on to posting #1058 on Dysfunctional Illusions of Rigor: Part 1 - Basic Illusions.

*HOWEVER*, it turns out that I inadvertently posted the entire article, including the advanced illusions, in all of posting #1058 - minus the set of references.  So here again is the second half of #1058, this time with the complete set of references.  Sorry for the confusion.

Note that postings #1058-59 are from Chapter 10, Dysfunctional Illusions of Rigor: Lessons from the Scholarship of Teaching and Learning, by Craig E. Nelson, Indiana University, in the book, To Improve the Academy: Resources for Faculty, Instructional, and Organizational Development, Volume 28, Linda B. Nilson, editor and Judith E. Miller, associate editor. Copyright 2010 by John Wiley & sons, Inc. All rights reserved. Reprinted with permission. Published by Jossey-Bass, A Wiley Imprint 989 Market Street, San Francisco, CA 94103-1741- www.josseybass.com

Regards,

Rick Reis
reis@stanford.edu
UP NEXT: Awake, Accountable, and Engaged

                                          Tomorrow's Teaching and Learning

            ------------------------------------------ 2,123 words -------------------------------------

                            Dysfunctional Illusions of Rigor: Part 2 - Advanced Illusions

Producing Brighter and Harder-Working Students in a Flash

In this section, we examine four more widespread illusions of rigor that are somewhat more "advanced."

                                                  Some Key Findings

In the previous section I focused on studies that have produced numerically powerful results. Equally important and impressive results have come from narrative traditions. Rose (1990) offered stunning examples of the barriers to students from "America's underclass" that result from faculty implicitly or explicitly assuming that the students have already mastered an array of disciplinary conventions before they arrive at college. (I regard Chapters Seven and Eight as essential reading for faculty.) Colomb (1986) found that one of the hardest tasks in learning to write for college (and work) was learning to avoid all of the perfectly reasonable things that one might say or write that are not allowed by the conventions of the discipline.

In biology, remarks on memories evoked by the colors of the chemicals used are out of bounds, as are comments indicating empathy for the lettuce or fruit flies that one is grinding up. Conversely, in humanities it is rarely appropriate to speculate on how different a visual piece would seem if we had, like many birds, four rather than three pairs of contrasted primary color responses. In either case, it may also seem digressive to wonder about any environmental racism involved in the extraction of the minerals that were used to produce the chemicals or pigments-even though exactly such considerations might be central to some courses in other departments.

                                                    Models for Change

Streepey (in Nelson, 1996) taught her classes how to write essay questions. She had them compare various B answers she had written for a question and then construct ideal answers individually and in groups. In one hour, she converted an average English section to a high-achieving one. Similarly, Walvoord and Anderson (1998) had students use rubrics to rate alternative examples prior to using those rubrics in actual writing.

These studies seemed to me to clearly support fundamental changes. But, I still was initially loath to use class time to teach students how to read and write appropriately.

Ultimately, I found that four additional illusions had blocked my progress.

Dysfunctional illusion of rigor 4. Students should come to us knowing how to read, write, and do essay and multiple-choice questions.

I was especially appalled when I saw that students did not know how to do multiple-choice questions in my introductory biology course. How, I wondered, could they have possibly graduated from high school and made it into Indiana University without knowing how to do multiple-choice questions? It took me some time to see that university level exams included a much greater emphasis on conceptual understanding, applications, and synthesis than was likely to have been possible early in high school when students typically take biology. I was similarly incredulous when I saw that about 90 percent of the students in my first-year seminars could not easily answer an essay question that required them to summarize the author's argument. This was true even when they were directed to read the two pages on which the argument occurred while working on the question. It became evident that students were used to saying what the text was about but not used to being able to accurately summarize the arguments made in the book. Clearly, they needed to learn to summarize the arguments before they were going to be able to learn to evaluate them.

More realistic view. Each of us needs to teach our students how to read pertinent materials and evaluate arguments and evidence. We need to teach this interactively in class, not just explain them. Because each discipline has its own conventions for how to read a book, how to write papers, what makes a great essay question answer, and more, we each have to do this repeatedly in different courses. I suspect that most students who are ready to start college without such help learned these skills in multiple AP courses.

Dysfunctional illusion of rigor 5. Traditional methods of instruction are unbiased and equally fair to a range of diverse students of good ability.

When I attended my first workshops on cultural and other biases in college teaching, I was shocked at the idea that courses such as calculus, physics, and biology were thought to be anything but nearly fully objective in both content and pedagogy.

More realistic view. Traditional methods of instruction favor students who have had multiple AP courses
and have otherwise had the exceptional preparation for college offered by elite high schools. In addition,
many or most such students come from well-off families, families that also have high expectations for academic success.

Rose (1990) convinced me that unintended discrimination is inherent in any assumption that students should come to us knowing how to read the way we want them to read, how to write the way we want them to write, and generally how to do the various tasks required to excel in our courses properly. Treisman's work (see above) convince me that even well-prepared students (high math SATs) are often disadvantaged by high school experiences that lead them to work alone. My own high school math teacher taught us that checking your homework with another student is cheating. It was a shock to find Treisman describing years later my solitary approaches to studying. It was an even greater shock to find him suggesting that if faculty didn't like the usual levels achieved by less-privileged students, they needed to build the social support required for learning.

Dysfunctional illusion of rigor 6. It us essential that students hand in papers on time and take exams on time. Giving them flexibility and a second chance is pampering the students.

More realistic view. Giving limited time flexibility on some assignments and a limited number of repeats on exams can be a way of fostering increased achievement and increasing fairness.

After I began to understand how standard classroom practices discriminated against students from less-privileged backgrounds, I asked myself what I was assuming when I gave an exam only once to a freshman biology class. It seemed that I was assuming that the student knew what it would feel like to have mastered the content at the university A level, that she had a realistic idea of how long this would take, and that she had control over her own time.

I hadn't understood that she might not have full control of her own time if, for example, she were a single parent with two children who caught the flu in the week before the exam, or if she had a real job and was ordered to take extra shifts to make up for someone who had the flu. Thus, the idea that favors privileged students, in the sense that it assumes things that are most likely to be true of traditional age students with limited other responsibilities.

I reluctantly decided that I should give each exam twice. Initially, so as to not to cut into coverage, I offered the second try in the evening at a time possible for everyone who wanted to take the exam. Students kept the better of the two grades. Performance improved markedly. I ultimately saw that studying twice for exams (which not every student did) taught on average more content than another lecture would have. I then started giving both exams in class time. Once this approach to exams proved successful, I adopted it in all courses (Nelson, 1996, in press-a).

I then asked myself whether I should continue to insist on rigid deadlines for other assignments. I ended up separating deadlines into two groups. Some were essential for my classes to function well. Preparation for discussion had to be done on time or the discussion would not work. I could allow limited flexibility on some other deadlines. Would it really matter if some lab reports were a bit late? On these, limited time flexibility might be appropriate. Perhaps lab reports would improve if students were allowed as many late days total as there were lab reports, with a penalty if the total were exceeded.

I have no evidence to support these practices beyond the fact that they worked for me and the feeling that they will obviously improve learning. I have found that many other faculty are fairly sure that they would also improve grades in their courses-and that like me, they initially are reluctant to sacrifice coverage or are worried that flexibility might lower standards. I suspect they will find that flexibility improves learning. Part of the change may be in student' attitudes. Students remarked that I had made it unusually clear that I really cared whether they learned and said that they consequently were trying harder.

Dysfunctional illusion of rigor 7. If we cover more content, the students will learn more content.

As evidence of my strong initial adherence to this view, I initially regretted each class period given over to an exam as a period in which I could not cover more of the important and fascinating biology. So much would have had to be left uncovered even if there were no exams.

More realistic view. The best courses are those that most successfully achieve the outcomes we see as most important. Initially, I was most strongly focused on content, especially on conceptual mastery.

The studies already discussed show that learning, student retention, and equity can be strongly increased by adopting active learning, by actively teachings students how to read and write within the framework of the course, and probably by allowing more flexibility on exams and deadlines. As I began to understand much of this, I realized with some dismay that I really was going to have to cover noticeably less material in class.

However, I stumbled on an approach that partially softened this blow, especially for course for advanced majors. I transferred part of the coverage to work outside of class time. I knew that even advanced majors tended to learn relatively little from reading assignments. I decided to try using more detailed study guides. These guides would be of a set of essay questions from which any exam questions over that reading would be drawn in whole or part, thus ensuring that the students paid attention.

I first set out to write all reasonable essay questions over one chapter. My goal was to list each question that I might have written after just assigning the students to read the chapter. I reached about fifty questions and was not yet done with the chapter. It was suddenly clear to me why  As on my exams typically had previously started at 70 percent when I included several questions over the readings. There was entirely too much material for the students to be expected to learn, and I had not been providing much guidance as to what was important. More appallingly, I realized that I had not decided what I most wanted to achieve by assigning the chapter. Making those decisions required substantial effort but deepened my understanding of my objectives. After the first few chapters, these tasks became easier.

Soon I was giving the students a set of about twenty essay questions over each chapter well before the exam. Often I told them that some parts of the text could be skimmed, skipped or read optionally. Most important, I often gave questions that asked for more careful analysis, synthesis, and critical thinking generally than I had been able to use previously. Even so, grades quickly rose: A's began at 90 percent. Thus I found that by using guided reading I could foster out-of-class learning to teach some key aspects of the content more effectively that when I had lectured on it. The fault lay not with my students but rather with my pedagogy. The new approach specified deeper and clearer learning objectives, gave substantial help in seeing how to reach them, and limited coverage both in lecture and by skipping parts of the text.

Even more realistic view. What I had come to gradually was an outcomes-based course design. Traditionally, we have chosen the most important content and covered it, hoping that the outcomes such as critical thinking would automatically result from learning the content. As alternative approach starts by selecting the outcomes that one most wishes to foster and then choosing the pedagogies, and finally the content that seems most likely to achieve these outcomes. The American Association of Colleges and Universities (www.aacu.org) has strongly advocated and effectively illustrated such intentional approaches to effective education. Key books now aid faculty in understanding and designing courses with these approaches (Bean, 1996; Diamond, 2008; Fink,2003; Grunert O'Brien, Millis, & Cohen, 2008; Mentkowski & Associates, 1999; Wiggins & McTighe, 2000).


REFERENCES for postings #1058 and #1059

Angelo, T.A., & Cross, K.P. (1993) Classroom assessment techniques: A handbook for college teachers
(2nd ed.) San Francisco: Jossey-Bass.

Barkly, E., Cross, K.P., & Howell Major, C. (2004). Collaborative learning techniques: A handbook for college faculty. San Francisco: Jossey-Bass.

Bean, J. (1996). Engaging ideas: The professor's guide to integrating writing, critical thinking, and active learning in the classroom. San Francisco: Jossey-Bass.

Colomb, G. G. (1986). Disciplinary secrets and th apprentice writer. Institute for Critical Thinking. Upper Montclair, , NJ: Montclair State College.

Copper, J. L., Robinson, P., & Ball, D. (Eds.) (2003). Small group instruction in higher education: Lessons from the past, Visions of the future. Stillwater, OK: New Forums Press.

Fink, L. D. (2003). Creating significant learning experiences: An integrated approach to designing college courses. San Francisco: Jossey-Bass.

Fillilove, R.E., &Treisman, P. U. (1990. Mathematics achievement among African American undergraduates at the University of California, Berkeley: An evaluation of the Mathematics Workshop Program. Journal of Negro Education, 59(3), 463-478.

Grnert O'Brien, J., Millis, B.J., & Cohen, M. W. (2008). The course syllabus: A learning-centered approach (2nd ed.) San Francisco: Jossey-Bass.

Hake, R. R. (1998). Interactive engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66(1), 64-74.

Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., Chang, A., DeHaan, R., et al. (2004). Scientific
teaching. Science, 304(5670), 521-522.

Jacobs, D. C. (2000). An alternative approach to general chemistry: Addressing the needs of at-risk students with cooperative learning strategies. Retrieved December 8, 2008, from http:/ems.earnegiefoundation.org/collections/castl_he/djacobs/index2.html
Mertkowski, M., & Associates. (1999). Learning that lasts: Integrating learning, development, and performance in college and beyond. San Francisco: Jossey-Bass.

Millis, B. J., & Cotell, P. G. (1997), Cooperative learning for higher education faculty. Phoenix, AZ: American Council on Education and Oryx Press.

Nelson, C. E. (1996). Student diversity requires different approaches to college teaching, even in math and science. American Behavioral Scientist, 40(2), 165-175.

Rose, M. (1990). Lives on the boundary: A moving account of the struggles and achievements of America's educationally unprepared. New York; Penguin. Science Education Resource Center. (2009). Teaching methods. Retrieved December 8, 2008, from http://serc.carleton.edu/sp/library/pedagogies.
html   

Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small -group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69(1), 21-51.

Terenzini, P.T., & Pascarella, E. T. (1994). Living with myths: Undergraduate education in America. Change, 26(1), 28-32.

Treisman, U. (1992). Studying students studying calculus: A look at the lives of minority mathematics students in college. College Mathematics Journal, 23(5), 362-372.

Walvoord, B. E. & Anderson, V. J. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.

Wiggins, G., & McTighe, J. Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.

----------------------------------------------------------------------------------------------------
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING
----------------------------------------------------------------------------------------------------