The Friday Institute’s Program Evaluation and Education Research (PEER) Group, in partnership with the Social Science Research Institute (SSRI) at Duke University, has begun a qualitative research study of summer programs directed by Senate Bills HB 82 and 196 and designed to mitigate the impacts of COVID on at-risk students. In order to better understand and to improve the impact of these programs, the Office of Learning Recovery and Acceleration (OLR) identified Elementary and Secondary School Emergency Relief (ESSER II) monies to enact four North Carolina university research partnerships. NC State University received a $120,000 grant from the North Carolina Department of Public Instruction (NCDPI) as part of a competitive process to support this collective research effort.
The yearlong project, titled “When Summer Learning Succeeds: Context, Conditions, and Strategies that Maximize Impact,” will allow researchers to better understand strategies and implementation conditions of high- and low-performing elementary school extension programs, with the goal of identifying effective program components and the conditions necessary for their success.
“As a former elementary teacher, husband to a current kindergarten teacher and father of two school-age children, understanding how to help young students recover from the impacts of the pandemic is more than just a professional and intellectual endeavor for me,” said Shaun Kellogg, interim executive director of the Friday Institute, senior director of the PEER Group and principal investigator. “Young students were disproportionately impacted by disruptions to in-person instruction and understanding not just which school and district programs helped students recover, but also why some were successful and others struggled, is critical to ensuring all students benefit from these programs.”
Building upon results from competency-based assessments administered to students in grades K-8 at the beginning and end of the program, this qualitative study will examine the implementation of elementary school summer programs targeting at-risk students following the 2020-21 and 2021-22 academic school years. Specifically, this study has three main goals:
- Explaining why some public school units (PSUs) were successful in helping students recover academically while others struggled.
- Exploring non-academic outcomes for students, teachers and families to develop a richer picture of impact and to guide future studies.
- Providing the Office of Learning Recovery and Acceleration (OLR) and PSUs recommendations to improve future programs.
“The pandemic has set many students back in their educational progress, exacerbating inequities that already existed and making dedicated attention to learning recovery absolutely essential,” said Jessica Sperling, director of applied research, evaluation and engagement at SSRI. “We also know that the 2020-21 year was particularly unique and challenging but that these challenging contexts can allow for ingenuity among educators. We are interested in learning how school and district experiences in 2020-21 have informed their 2021-22 planning, as well as the strengths and assets that we may be able to identify and elevate.”
The study will be conducted in multiple phases using a comparative case study design focused on program implementation and impact in State Board of Education (SBE) Regions 3 & 4 of North Carolina. The four-phase comparative case study design allows for further exploration of the factors that helped students recover some academic skills lost due to COVID, as well as any non-academic skill gains (e.g., social skills, emotion management skills) students may have made over the summer months.
“The first phase of the study included reviewing summer program applications and student assessment data, which allowed us to take a wide lens to see how much progress students made during summer 2020-21,”said Alicia Fischer, research associate in the PEER Group at the Friday Institute. “Once we had the understanding of how students performed on pre- and post-tests on reading and mathematics, Phase II includes talking with administrators and teachers with PSUs that had high-performing sites, low-performing sites, or both. We want to understand both the variations in student performance within districts, as well as across districts, from both a district-wide administrative level and a site-specific teacher perspective.”
Phase III of the study includes targeted site visits to observe summer programming, as well as to get a deeper perspective of the efficacy of the programming from parents and students. The final phase of the study will include a comparative analysis of summer programming from year one to year two, highlighting unique findings that contribute to student successes. The final phase will also include information dissemination to inform best practices moving forward.
“On behalf of the Office of Learning Recovery and Acceleration, we are excited to see the outcomes of these qualitative studies,” said Erin Manuel, research analyst in the OLR at NCDPI. “We know the research results will significantly contribute to the educational field and further our understanding of summer programs’ collective experiences and implementation. These studies will inform our promising practices across the state and help make evidence-based recommendations to practitioners and policymakers.”