Generalizability of a Technology-Based Intervention to Enhance Conceptual Understanding in Mathematics

Abstract

Three previously reported experiments found that a technology-enhanced intervention increased student conceptual understanding of mathematics in Texas. To investigate generalizability to broader populations and settings, we triangulate among three methods. First, we examine interactions between demographic variables and intervention effects. We found that the intervention was not sensitive to typical variations in school populations. Second, we use propensity score methods to measure the match between the sample and a broader population. The sample matches the school population in Texas, with minor exceptions; we report adjusted effect sizes. Third, quasi-experimental research with populations outside of Texas are considered. Results from Florida and England were consistent with Texas findings. Across three methods, the results suggest that the experimental findings generalize across populations and settings. This work also establishes a practical approach to investigating generalizability in experimental research in schools.