"Participants who used an iPad exhibited significantly higher transfer learning scores compared to traditional textbook readers. Gertner found that those who used an iPad scored higher on transfer learning than those using a traditional text. Gertner suggests that the integration of multimedia elements (i.e., the organization and presentation of content) may lead to greater gains on transfer learning scores."

Tomorrow's Professor Msg.#1255 Do iPads Affect Reading Comprehension and Learning?: The Jury Remains Out

 

Folks:

The posting below looks at a study that compared various kinds of reading comprehension using an iPad vs a traditional textbook. It is prepared by the Research and Evaluation Team, Office of Information Technology,(http://www.oit.umn.edu/research-evaluation/). University of Minnesota - Twin Cities. http://z.umn.edu/research. In an effort to make research in the educational technology field more accessible, OIT\\'s Research & Evaluation team produces frequent brief synopses of important recent studies. These synopses may be freely shared and used for non-profit academic purposes. http://z.umn.edu/briefs. For further information contact Dr. J.D. Walker (jdwalker@umn.edu).

Regards,

Rick Reis
reis@stanford.edu
UP NEXT: Teaching Quantitative Reasoning

Tomorrow's Teaching and Learning 
--------------------------------------------- 1,082 words ---------------------------------------------
Do iPads Affect Reading Comprehension and Learning?:  The Jury Remains Out
"Always-on," Internet capable mobile devices such as smartphones and tablet devices (for example, iPads), are increasingly prevalent at institutions of higher education (IHEs). While the popularity of the devices is undisputed, questions remain regarding the effects of such devices on learning in higher education. Abilene Christian University's (ACU)[http://www.acu.edu/] "iPad Studies" [http://www.acu.edu/technology/mobilelearning/research/ipad-studies.html] are examples of an IHE conducting research on the effects of mobile devices on learning. A recent study by Gertner [http://www.acu.edu/technology/mobilelearning/documents/research/effects-of-technology-on-learning.pdf] examined the use of the iPad as a multimedia device and its effectiveness as a means of improving reading comprehension and transfer learning.

Study Design

Gertner employed two separate theoretical constructs in his study. The first was Cognitive Theory of Mulitmedia Learning (CTML), which states cognitive visual overload is troublesome for learners. However, when visual content appears in tandem with verbal cues (e.g., narration) cognitive overload is reduced, thus increasing the potential for a learner to perform tasks more effectively. Transfer theory, the second construct utilized by Gertner, is defined as "the effect of prior learning on new learning or performance" (Mayer, 2011). For example, if one knows the functions of parts of an automobile engine, one might better be able to diagnose and solve problems related to engine performance. Both of these theories support Gertner's hypothesis that comprehension and transfer would be higher for those using an iPad.

To test these hypotheses, Gertner randomly assigned 69 participants from an Introductory Psychology class at ACU to read a chapter from a course text using an iPad with an e-reader app (n=25) or a traditional textbook (n=44). Gertner gave the participants a comprehension and transfer test following the reading session. The main points from Gertner's study follow.

Findings

1. Participants who used an iPad did not score higher on reading comprehension than participants who used traditional textbooks. Gertner found no statistically significant difference in reading comprehension between the groups (iPad versus traditional text). He concluded those using an iPad performed comparably to those with a traditional text (more on this claim Discussion & Critique, below).

2. Participants who used an iPad exhibited significantly higher transfer learning scores compared to traditional textbook readers. Gertner found that those who used an iPad scored higher on transfer learning than those using a traditional text. Gertner suggests that the integration of multimedia elements (i.e., the organization and presentation of content) may lead to greater gains on transfer learning scores.

Discussion

Study Design.

* Random Assignment of participants. Participants were divided into two separate groups: one group using traditional textbooks and another group using iPad e-text books. Random assignment of participants can be challenging to execute, so this aspect of Gertner's design makes it truly experimental. Such designs are not common in higher education research.

* Instrument Validation. Gertner noted assessment measures were not validated prior to the study. This is highly problematic due to the fact Gertner's instruments might not be measuring what he claims to measure (in this case, comprehension and/or learning). This threat to the internal validity of the study, termed "instrumentation," creates the possibility that any differences between the two groups are simply an artifact of the tool being used to evaluate any of the outcomes.
    
* Non-blind scoring. Gertner stated he knew which participants were in each group (iPad or traditional). As such, this knowledge may easily have biased the results of the study. In addition to bias, reliability becomes a concern as well. These are especially important points to consider since Gertner was the sole scorer of the assessment measures.

* Pretest for academic ability. Due to the study's design, we do not know whether the effects of the device were really that of the device or another factor. It could be those in the iPad group had higher cognitive abilities to begin with and thus would perform better on the learning or comprehension test. If the sample size were larger, then chances increase for statistical equivalency between the two groups (due to random assignment) would be better, thus eliminating the need for a pretest. However, because no controls or pretest were in place, it is difficult to conclude whether the transfer learning (or reading comprehension) was due to the iPad and not an artifact of some other factor, such as differing academic ability.

* Selection of ANOVA for post-hoc analysis. Although the use of a one-way ANOVA in this study was appropriate, the author could have done more with his analyses. For example, although Gertner collected participant demographic data, he did not include these data in his analyses.  It would be useful to explore these data in post-hoc analyses to determine if any differences exist by gender, race, etcetera, within the groups.  Such analyses would offer the reader more specific information about the effects of mobile devices on learning outcomes. Moreover, it would allow the researcher to provide more precise recommendations to higher education constituents based on his findings.  

Critique

Regarding the first finding, Gertner suggested that while there were no statistically significant differences between groups on reading comprehension scores among those using iPads, they are at least comparable to traditional textbooks. While that is not an inappropriate statement, he goes on to conclude, "This trend in the data is favorable to those who advocate greater e-text usage in educational settings" (p. 27).  This also overstates the case and makes a claim that is unsupported by the data (i.e., this is a single study and that does not demarcate a 'trend' of any kind towards the efficacy of increased usage). Given the threats to internal and external validity of Gertner's study, extreme caution should be utilized in concluding that these findings are generalizable to other populations.  This link is somewhat misleading, as comparability does not necessarily lead to favorability.

Regarding the second finding, Gertner speculated that the reason participants scored higher using the iPad than traditional text was perhaps due to the functionality of the iPad (e.g., appearance of content, key word searchability) and the way content appeared on the iPad. This may be true; however, due to the lack of any evidence to support this assertion, it is difficult to determine whether this claim has merit.
 
Conclusions

A (very) cautious step. While Gertner's study contains some methodological shortcomings, his study is important, as it opens the conversation for further investigation into the use of mobile devices and important student learning outcomes. That said, future studies should carefully adhere to more rigorously focused and controlled methodologies in an effort to provide more systematic, valid, and generalizable (i.e., tangible) results.

References

Gertner, R. (2012). The Effects of Multimedia Technology on Learning, (Unpublished Master's Thesis).[http://www.acu.edu/technology/mobilelearning/documents/research/effects-of-technology-on-learning.pdf]  Abilene Christian University.

Mayer, R. E. (2011). Applying the science of learning. Boston, MA: Pearson.

----------------------------------------------------------------------------------------------------
TOMORROW'S PROFESSOR MAILING LIST
Is sponsored by the STANFORD CENTER FOR TEACHING AND LEARNING
----------------------------------------------------------------------------------------------------