Exploring the Relationship Between Continuous Improvement Culture and Afterschool STEM Program Quality

Description of project work

In partnership with the University of North Texas, SRI Education is conducting a 5-year study, funded by the National Science Foundation (#1906490), to investigate continuous quality improvement (CQI) practices and STEAM programming in California’s publicly funded afterschool programs. The study, called CI-STEM, began in summer 2019.

The study team is examining three areas of interest: the scope and impact of California’s expanded learning CQI policy, implemented by the Expanded Learning Division in the California Department of Education; the scope and content of STEAM programming in the state’s system of more than 4,500 publicly funded afterschool sites; and the impacts of a cultural change effort, led by the Expanded Learning Division and the Center for Systems Awareness, on CQI and STEAM offerings within the afterschool system. Working with InSites, we are using complex systems theory to understand the dynamics of the afterschool system at different levels and to generate explanations for patterns in CQI implementation and STEAM programming in the afterschool sites.

California’s publicly funded afterschool programs stand out in many ways in terms of size, scope, level of dedicated state funding, and sheer diversity of people and places. The state’s effort to make CQI a way of life at such a large scale is unparalleled. Additionally, the state’s effort to bring about sustainable change in mindsets and organizational culture through systems thinking and applying a compassionate systems framework is a unique experiment in education reform.

The study team is documenting how these efforts unfold over time to provide insights about systems change, organizational culture change, and the role of systems thinking in CQI in California’s afterschool system. We also aim to understand which specific CQI resources, tools, and practices work well in supporting ongoing improvement in afterschool STEAM programming, as well as which work best in different conditions, contexts, and communities.

The study involves three research activities:

• System study interviews. The team is in the process of mapping the expanded learning system in which CQI is implemented and how implementation of the compassionate systems framework changes practices and mindsets to better support CQI. This work involves interviews with leaders and professionals in the Expanded Learning Division, the Center for Systems Awareness, the Statewide System of Support for Expanded Learning, and intermediary organizations supporting these efforts (such as the California AfterSchool Network, ASAPConnnect, California School-Age Consortium, Partnership for Children & Youth, and Temescal Associates). We are also interviewing grantees, site coordinators, frontline staff, and others.
• Surveys. The team plans to survey frontline staff, site coordinators, and grant managers or program directors throughout California to describe organizational culture, CQI processes, and informal STEAM offerings. We will launch the surveys in the 2021–22 school year look at how patterns change over 3 years.
• Case studies. The team will conduct in-depth case studies of CQI practices and STEAM offerings in afterschool programs statewide to understand how sites enact CQI and STEAM locally. We will observe activities and meetings, interview staff and youth, review documents and artifacts, and connect these to insights about the system and patterns from the interviews and surveys.

This project is supported by a grant from the National Science Foundation (#1906490). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Associated SRI team members