Early Childhood Classroom Observation (ECCO) Study

SRI Education is conducting a study to better understand how video recordings can support the measurement of high-quality instruction in preschool classrooms. The Early Childhood Classroom Observation (ECCO) study focuses on two common measures of early learning programs: the Classroom Assessment Scoring System (CLASS) 2nd Edition and the Early Childhood Environment Rating Scale, 3rd Edition (ECERS-3). For each measure, we are conducting in-person (live) observations while the preschool classroom is simultaneously video-recorded. We will compare the data from these two observation methods to see how well they agree with each other (interrater reliability) and to better understand their similarities and differences. This work is funded by the Bill & Melinda Gates Foundation.

To address issues with equity and generalizability, we will explore differential item functioning for different groups of children. We will start by looking at the differences between teacher and classroom demographic groups and examine differences in score distributions across these groups to determine if there is any bias in scores. In addition, we will compare psychometric properties for the in-person versus video-based observations to examine differences in score distributions for these two methods. Finally, we will use multivariate regression to determine if any characteristics are statistically significant predictors of the observation measure scores.

As part of the study, we are gathering teacher, program leader, and other stakeholder perceptions of the challenges and benefits of using video observations through interviews and surveys. We are also documenting information on the logistical considerations for conducting video-based observations, including necessary video equipment and costs of live and video-recorded observations. The results of this reliability study will provide policymakers, early childhood educators, and researchers with information about the trade-offs for each method of data collection.

Our work