""George D. Kuh, in his brief pamphlet, High-Impact Educational Practices, advocates a data-driven, incremental approach. His work, reviewed here by Jennifer D. Berg, emphasizes five particularly successful practices, and it begins the process of exploring the evidence for their success. In the end, Kuh's assessment is that much work remains to be done in developing and understanding what makes for good teaching." Sean C. Goodlett and Matthew Johnsen "
Tomorrow's Professor Msg.#1275 What’s “Best” in Our Practices? (in teaching and learning)
The posting below is a review by Jennifer D. Berg, assistant professor of Mathematics at Fitchburg State University, Fitchburg, Massachusetts, of the booklet, High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter, by George D. Kuh. It is from Currents in Teaching and Learning, Vol. 4, No.2, Spring 2012. Currents in Teaching and Learning is a peer-reviewed electronic journal that fosters exchanges among reflective teacher-scholars across the disciplines. It is a publication of the Center for Teaching and Learning of Worcester State University, Worcester, Massachusetts, U.S.A. Copyright © WSU, 486 Chandler Street, Worcester, MA 01602. All rights reserved. Reprinted with permission.
UP NEXT: Making Your Research Paper Discoverable: Title Plays the Winning Trick
Tomorrow's Teaching and Learning
-------------------------------------- 1,577 words --------------------------------------
What's "Best" in Our Practices? (in teaching and learning)
High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. By George D. Kuh with an introduction by Carol Geary Schneider and findings on student success from AAC&U's LEAP initiative. Association of American Colleges and Universities, 2008, 35 pp., $25, ISBN 978-0-9796181-4-7.
Faculty searching for reliable research on new teaching methods to help students engage with learning run into innumerable obstacles. Chief among these are the plentiful claims of "best practices," most of which have little or no supporting evidence. Indeed, a sizeable portion of such research done at the university level examines a single institution's program(s), while the analysis of the evidence deployed rarely yields results applicable to one's own institution. George D. Kuh seeks to overcome the gaps in the scholarship. In recent years, the widespread use of the National Survey of Student Engagement (NSSE) has produced a tremendous amount of data to aid in the search for what Kuh calls "high impact practices." In his brief essay of the same name, Kuh deploys NSSE data to support the claim that ten practices are particularly effective. These practices, he insists, should be more widely used and more thoroughly researched.
The practices Kuh identifies are first-year seminars, common intellectual experiences, learning communities, writing-intensive courses, collaborative assignments, undergraduate research, global/diverse learning, service/community learning, internships, and capstone courses/projects. The ideal version of each of these experiences is briefly described, with the caveat that "these practices take many different forms, depending on learner characteristics and on institutional priorities and contexts" (p. 9). All the practices have been investigated in the NSSE, either directly or indirectly. Indeed, they were included in the survey, in part, because evidence existed to suggest that they were associated with "important college outcomes."
As a founding director of the National Survey of Student Engagement and the Chancellor's Professor of higher education at Indiana University- Bloomington, George Kuh is well positioned to offer reflections on the data collected over the ten years that the survey has been administered. He is also qualified to speak to the significance of the data. NSSE is used at hundreds of institutions, and since 2000 its website has recorded some 3.7 million responses. The survey asks students about their participation in programs and the time spent on activities. Kuh is also a leadership council member of the American Association of Colleges and Universities' (AAC&U) new LEAP initiative, Liberal Education and America's Promise. As part of their work on LEAP, the AAC&U identified-through multi-year discussions with faculty, employers, and accreditors-a set of "student learning outcomes that almost everyone con- sidered essential" (p. 3).
Ultimately, Kuh wants to know what university faculty can do to increase student engagement. The essay begins by setting out the evidence collected from NSSE as it relates to five of the ten practices. The author also describes the main aspects of these practices that he feels are critical to increasing student engagement. He then examines the data more closely to determine the effects these practices have on students from underserved populations. The pamphlet closes with Kuh's suggestions on how colleges should use his findings at their institutions.
Kuh examines two sets of data. The first set shows a positive relationship between students who report that they have participated in one of the five practices he has selected and their responses to other items in the survey. These "other items" are grouped together into eight clusters: deep learning, gains general, gains personal, gains practical, level of academic challenge, active and collaborative learning, student-faculty interaction, and supportive campus environment. Service learning and student-faculty research show the most promise, study abroad the least.
The second set of data breaks out both student ethnicity and pre-collegiate achievement levels and compares participation in "educationally purposeful activities" (p. 18) with first-year GPA and probability of returning for the second year of college. In each of the cases there is not only a positive gain for all students but a greater effect on the students from the underserved populations. "Educationally purposeful activities" refers to a cluster of 19 items on the survey, including having: made a class presentation; prepared two or more drafts of a paper assignment before turning it in; discussed grades or assignments with an instructor; received prompt feedback from a faculty member on your academic performance (written or oral). Kuh then identifies the aspects of each of these practices that makes them "unusually effective." Here the reasons range from "students typically get frequent feedback about their performance" to "can be life changing" (p. 17). This section gets little support from the NSSE data; the survey does not inquire after the characteristics of these activities, and, as Kuh himself notes, there is considerable variance in the characteristics of each of these practices.
Kuh concludes the essay by arguing that all students should participate in at least two high-impact practices during their undergraduate careers, one in the first year and one later, when students are taking courses in their major. He goes on to argue that "common intellectual content should be a nonnegotiable organizing principle" for the early college experience (p. 19). Furthermore, he points out that it is not sufficient to develop activities for students that simply appropriate the names of the high impact practices. Educators must cultivate practices that aim for the ideals set out in the first part of the essay. To do so, more research is needed on what links student participation in these activities to better student learning.
While Kuh does an admirable job of extracting a handful of important themes from the tremendous amount of NSSE data, he does less well in his exposition of those themes. The transition from the ten practices listed at the beginning of the work to only five is never directly addressed and makes two important omissions: writing-intensive classes and global/diverse learning. In the Introduction to the work, AAC&U president Carol Geary Schneider points to a survey of employers of recent graduates. This study examines the preparedness of graduates in eleven learning outcome areas. Writing and global knowledge are two of the three areas where employers rank recent graduates as least well prepared. Almost all colleges have both writing and global learning requirements, so the omission of data related to these two practices here is disappointing.
Other weaknesses derive from the brevity of the essay. For instance, Kuh summarizes, in a page-long table, the percentage of students who report participating in learning communities, service learning, research, study abroad, internships, and capstone experiences. The information is broken down by institutional type (2005 Basic Carnegie, public/private sector, and Barron's selectivity) as well as by student characteristics (ethnicity, full/part-time enrollment, first-generation status, transfer status, and age). Unfortunately, Kuh spends almost no time highlighting the information from the table, except to point out that both first-generation and African-American students are less likely to participate in the five practices. Here the designation of "less likely" is not well defined and varies among the different practices; both of these sub-groups are as or more likely to participate in some of the practices.
Moreover, when Kuh discusses the reasons why the five practices are particularly effective in increasing student engagement, he often fails to support his ideas with evidence. Nor does he take the opportunity to discuss either the limited conclusions that can be drawn from NSSE data or the lack of research done on the details of such practices. NSSE data can at best highlight correlations. This is in part because of the wide range of educational practices all using the same name. More detailed research is needed for the claims of causation Kuh makes here. As it stands, readers may walk away from this work excited to develop some of these practices at their institutions, but they will find little guidance on how such opportunities should be designed. Here Kuh falls short of answering the question, "What can we do to increase student engagement?" and falls prey to the weakness of so many claims of "best practices" in higher education - focusing on style over substance.
The AAC&U has published a follow-up work called Five High-Impact Practices: Research on Learning Outcomes, Completion, and Quality (Brownell and Swaner, 2010), which reviews the research that has been done on these practices. In each case both the authors and Kuh himself, who wrote the foreword, claim that much more and much better research is needed on how and when participation in these practices will increase student learning.)
Drawing correlations from NSSE data between student participation in these practices and positive learning outcomes, as Kuh does here, should serve not as a stopping point, but as a call to further research on the practices. When a student marks down on the survey that he has "worked with a faculty member on a research project" it is unclear what aspects of that work were essential in improving the student's learning. Most of the practices mentioned here were included in the survey because there was early evidence to suggest they were likely to increase student learning. Instead of circularly reinforcing this suggestion with the broad evidence of NSSE, we should begin to flesh out why these practices help our students and then work to use those critical features in all of our teaching practices. The call for "data-driven decisions" is sound. We should demand that the data be adequate to guide our own development as educators. --
Brownell, J. E., & Swaner, L. E. (2010). Five high-impact practices: Research on learning outcomes, completion, and quality. Washington, DC: Association of American Colleges and Universities.
Jennifer Berg is Assistant Professor of Mathematics at Fitchburg State University. Her research interests include representation theory and mathematics education.
* * * * * * *
NOTE: Anyone can SUBSCRIBE or UNSUBSCRIBE to the Tomorrows-Professor Mailing List by going to:
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING