In higher education, if students fail to successfully complete the courses offered in the first few terms of school, they can jeopardize their dreams of a college degree. In two-year colleges, the courses representing the greatest hurdles are often developmental courses in English or math. In four-year colleges, they often are introductory courses in subjects such as biology, psychology and business, which serve as the “gateway” to the major.
To improve student outcomes in these courses, colleges and universities–along with leaders in education reform–are turning to adaptive learning courseware that uses computer algorithms to deliver personalized feedback and instructional activities for individual students. This courseware has the potential to help students through these challenging first-year courses. Of particular interest is learning whether this technology benefits low-income students, or if it creates an adverse impact through inequities between low-income students and their more affluent counterparts.
As Bill Gates laid out in his keynote at the recent Arizona State University (ASU) + Global Silicon Valley (GSV) Summit, the education field needs to build an evidence base for understanding which technology tools work, and recently, through funding from the Bill & Melinda Gates Foundation, researchers from SRI Education evaluated adaptive courseware’s impacts on higher education student outcomes. Called the Adaptive Learning Market Acceleration Program (ALMAP), the foundation provided grants to 14 colleges and universities to integrate adaptive courseware in a range of developmental and gateway courses. The goals of the study were to advance evidence-based understanding of how adaptive learning technologies can improve opportunities for low-income students to learn and complete postsecondary degrees, examine the cost impacts for institutions, and determine instructor and student satisfaction.
Over the course of this rigorous two-year evaluation, SRI researchers analyzed data from over 19,500 students in classes taught by more than 280 instructors. Keeping in mind the potential differences between students that could influence results, researchers established a baseline equivalence of students in the courseware and courses being compared in order to provide valid answers to questions about the relative effectiveness of the adaptive courseware.
Quasi-experiments were then conducted, in which side by side comparisons were made between courses that integrated adaptive learning courseware and those that were taught business-as-usual. The course delivery approaches were organized into three categories:
- Blended Adaptive vs. Lecture— adaptive courseware was used as part of a shift from traditional lecture to blended instruction;
- Online Adaptive vs. Online— adaptive courseware was used as an enhancement to existing online courses; and
- Blended Adaptive vs. Blended—adaptive courseware was swapped into face-to-face courses already using blends of classroom-based and online approaches to support learning.
SRI’s research found that adaptive courseware holds promise for improving some student outcomes. For example, switching from a lecture format to adaptive blended instruction had a positive impact on student learning as measured by posttests. In addition, the evaluation found that impacts for low-income students were equivalent to those for students overall, which contrasts with some previous studies that have suggested that online and blended learning approaches put low-income students at a disadvantage.
As colleges and universities continue to grapple with cost issues such as cutbacks and decreased funding, learning technology needs to be affordable and show value and sustainability in order to justify investment. SRI researchers found in most cases, the first year of implementing adaptive learning courseware led to higher costs—largely due to instructor training and the initial outlay of implementation. However, after the second year, adaptive courseware generally had lower ongoing costs than the business-as-usual versions of the same course.
Finally, SRI researchers looked at how satisfied the instructors and students were with the adaptive courseware in both two-year and four-year environments. Initial results suggested that instructors and students in two-year colleges embraced the courseware somewhat more than their counterparts in four-year colleges. This work is being continued through another SRI Education research effort sponsored by the Bill & Melinda Gates Foundation, Next Generation Courseware Challenge.
The ALMAP evaluation report provides a glimpse into the state of adaptive learning courseware implementations across the range of colleges and universities in the United States. In reviewing the findings, it is important to keep in mind that learning technology—along with how students and instructors are using the technology—is rapidly evolving, and future research in this field is essential to further understand the impacts of adaptive courseware. However, this evaluation provides a first step toward cultivating a critical community of inquiry around the usefulness of new courseware in supporting student learning and course completion in institutions of higher education.
