Initial Efficacy Study of Data Wise

SRI International, together with the Harvard Graduate School of Education and Koru Strategy Group, is conducting the Initial Efficacy Study of the Data Wise Improvement Process (Data Wise). Data Wise supports educators in using collaborative data inquiry to drive continuous improvement of teaching and learning. This study examines the impact of Data Wise on upper elementary mathematics achievement, mathematics instructional practice, instructional teams’ collaborative data practices, and teachers’ mindsets. In addition, the study measures the fidelity with which study schools implement Data Wise practices, the sustainability of Data Wise practices within instructional teams, the long-term effects of Data Wise for students, and the cost-effectiveness of Data Wise.

Teaching is a challenging, complex profession with high stakes for students’ future academics, careers, and civic engagement. In recent years, many districts have turned to data-driven decision-making and continuous improvement strategies that rely on using data to support teachers in professional development. But these programs vary in sophistication, approach to data use, and ease of use. And few, if any, have the kinds of evidence required by the Every Student Succeeds Act.

Data-driven decision-making

One of the more promising approaches to data use is data-driven decision-making. In this approach, instructional teams collaborate around multiple forms of data to help teachers shift their practice and adopt new teaching strategies based on evidence of what students did and did not learn. When data-driven decision-making is done well, teams include multiple perspectives: teachers, who are closest to the work and will implement any desired instructional changes; coaches or other instructional experts, who support these instructional changes; and school leaders, who can align the team’s work to the school’s needs and priorities. To be successful, data-driven decision-making requires collaboration, shared practices, and constructive critical dialogue.

The Data Wise approach to collaborative data inquiry and action

Developed at the Harvard Graduate School of Education, Data Wise is a structured but flexible continuous improvement framework that supports instructional teams in collaborating around multiple sources of data, surfacing evidence of their students’ strengths and opportunities for growth, identifying shared problems of practice, and pursuing evidence-based solutions. Data Wise’s job-embedded tools are designed to step teams through a process that fosters collaboration and results in meaningful changes in instructional practice. By using an inclusive approach that provides all educators with a voice in reviewing, interpreting, and acting on data, Data Wise engages teachers in decision-making instead of requiring them to implement changes handed down by others.

Data Wise has four core components: (1) intensive professional development workshops for foundational learning; (2) protected meeting time for instructional teams to engage in collaborative data inquiry; (3) job-embedded tools for each step of the inquiry process; and (4) monthly virtual coaching for teams on holding effective collaborative meetings, using the job-embedded tools to complete inquiry cycles, and action planning and assessment of progress.

Data Wise’s reach and previous research

Since the inception of Data Wise in 2006, more than 4,000 participants have received the training directly from Harvard through the Data Wise Leadership Institute (intensive onsite workshops for teams) and the Data Wise in Action Program (monthly virtual coaching during the academic year). Over 100,000 educators have registered for the Data Wise Massive Open Online Course (MOOC). And since the 2018–19 school year, Koru Strategy Group’s national network of Certified Data Wise Coaches have supported more than 200 schools in learning and implementing the Data Wise process.

Data Wise has been the subject of multiple, rich case studies to understand its implementation. However, there have been no previous evaluations of the impacts of Data Wise on teachers and students.

This study

In 2020, the Institute of Education Sciences awarded SRI Education an initial efficacy grant to provide evidence of the implementation, impacts, costs, and sustainability of Data Wise. This study will take place during the 2022–23 and 2023–24 school years. Fifty schools will be grouped into either a treatment condition (receiving Data Wise training and coaching) or a control condition (continuing with business as usual). Koru will provide the Data Wise training and coaching to grade 4 mathematics teams at the Data Wise schools.

SRI will collect the following implementation, impact, and cost data:

Direct observation of mathematics instructional practice to understand instructional practice
Instructional team meeting logs to understand teams’ meeting practices
Instructional surveys to understand teachers’ and instructional leaders’ mindsets and practices
Implementation fidelity to understand whether Data Wise was implemented as intended
State standardized mathematics test scores to track student achievement
Measures of cost to estimate cost-effectiveness

The findings

The study team anticipates final results will be publicly available in summer 2025. The team will share preliminary results with project partners, including participating school districts, before releasing the final results to the public.

Data Wise: https://datawise.gse.harvard.edu/

Koru Strategy Group: https://www.korustrategy.com/