Summer Learning Program Quality Intervention
In 2013, the David P. Weikart Center (Weikart Center) and National Summer Learning Association (NSLA) began a collaboration to improve the quality of summer learning programs. The Summer Learning Program Quality Intervention (SLPQI) is a continuous improvement intervention for summer learning programs that includes four core components: (1) a standard and measures for quality of management and instructional practices – and the Summer Learning Program Quality Assessment (Summer Learning PQA); (2) training and technical assistance supports, (3) performance data products and (4) a continuous improvement cycle that fits the prior three elements to local circumstances and resources. The SLPQI and the Summer Learning PQA were designed to advance the science and practice of continuous improvement by focusing on qualities of learner experience that optimize skill building in specific domains, active-participatory and academic.
What’s the background for this project?
Beginning with an initial pilot phase in 2013 and continuing with a second phase in 2014, the David P. Weikart Center for Youth Program Quality (Weikart Center) and the National Summer Learning Association (NSLA) began a formal collaboration to address summer learning program quality measurement and improvement. Summer learning programs with a focus on improving specific academic skills are an increasingly important part of the expanded learning landscape. Numerous studies show significant summer learning loss that disproportionately affects disadvantaged students, which means summer programs are positioned to play a key role in youth and child development. While a there are clear indications that summer learning programs can be effective, few rigorous studies have looked closely at how staff practices can affect summer learning participation and relate to school success outcomes. The Weikart Center and NSLA seek to bring a customized, effective, and engaging set of measures and supports to enhance the ability of the summer learning field to implement intentional quality improvement efforts and lay the groundwork for studying the connections between summer program quality and child and youth outcomes.
Why a new assessment tool?
Summer programs pose a unique challenge for quality assessment and improvement. The difference in program duration alone (roughly 37 weeks for afterschool and 4-12 weeks for summer) creates a set of challenges for program planning, design, and implementation when comparing summer to school-year programs. In an effort to find a balance between the need to stem summer learning loss and the desire to provide a fun, enriching environment of exploration, high-quality summer learning programs blend youth development strategies into intentional learning activities. The Summer Learning PQA is specifically designed to measure the quality of staff instructional practices in both enrichment and academic settings as well as quality management practices that set the stage for successful staff engagement in summer programs. Substantial overlap with both the Youth and School-Age PQA also provides a way for programs to seamlessly transition from school-year to summer quality improvement efforts.
What does the process look like?
The Summer Learning PQI is modeled after the YPQI. It is an intervention comprised of a series of continuous improvement practices, which in this last phase of the pilot were designed to be program planning, quality assessment, improvement planning, and coaching by site managers during staff instruction. As demonstrated in the YPQI study , high fidelity to these practices should result in improved instructional practice, which in turn results in an improved experience for youth participants. In order to accommodate the short summer schedule, the Summer Learning PQI is built to remove the burden of data collection and reporting from program staff by relying on trained external assessors. The data collected by assessors results in a performance data packet that has both a summary report meant for immediate improvement while a program is in session, as well as a full report for longer term improvement planning.
What does the Summer Learning PQA Measure?
High quality summer learning programs must first of all be high quality youth programs. The basic structure and standards of the research-validated Youth PQA are present in the Summer Learning PQA as foundational measures of program quality. In order to enhance the ability of the tool to measure broader programmatic elements that are unique to summer, elements of NSLA’s Comprehensive Assessment of Summer Programs (CASP) are also integrated into the new tool. With this understanding that effective summer programs employ positive youth development practices in addition to incorporating academic skill-building, the Summer Learning PQA contains the scales indicated in the list on the right. Those who are familiar with the Youth and School-Age PQAs will notice that there are new scales reflecting positive youth development-based standards for academic instruction, particularly in the Engagement and Academic Content Area domains.
How has this work been supported?
A group of funders came together to support both phases of the pilot. To date, the work has been supported by: The Wallace Foundation; The David and Lucille Packard Foundation; The Raikes Foundation; The Doug and Maria DeVos Foundation; The William T. Grant Foundation.
Where has the tool been used?
In 2013, during the first phase of the pilot, the Summer Learning PQA was piloted in three cities: Baltimore, MD; Grand Rapids, MI; and Oakland, CA. A total of 16 program sites participated in the initial process. In 2014, during the second phase of the pilot, a revised version of the tool and full Summer Learning PQI process was implemented in two new cohorts - Seattle, WA and Northern CA – with several phase one sites returning for a total of 32 program sites.
What kind of research has been done so far?
In Phase I (2013), Weikart Center conducted a proof-of-concept pilot to design the SLPQA standard/measure and explore application in a continuous improvement cycle at 16 sites in Baltimore, MD, Grand Rapids, MI and Oakland, CA. Additional detail is available in the report, Summer Learning Program Quality Assessment: 2013 Phase I Pilot Report (Ramaswamy, Gersh, Sniegowski, McGovern, & Smith, 2014). The Phase II study (2014) explored feasibility and fidelity of the SLPQI when implemented at scale in summer systems in Seattle, WA, Northern California and Grand Rapids, MI. The report, Summer Learning Program Quality Intervention (SLPQI): Phase II Feasibility Study, provides additional detail (Smith, Ramaswamy, Gersh, & McGovern, 2015). During Phase III (2015-2016), we are focused on implementation at scale with an emphasis on exploring how to best implement and support a full, city-wide, multi-year SLPQI with high fidelity. The interim report for this phase was produced following the first of two years of the study (Smith, Ramaswamy, Hillaker, Helegda, & McGovern, 2015). Phase III also included a quality-to-outcomes study in Seattle Public Schools which intended to describe the effectiveness of summer programs in terms of instructional quality and academic skill growth. Primary goals of the study were to (1) describe the effectiveness of SPS summer programs in terms of instructional quality and youth skills and (2) to accumulate further validity evidence for the Summer Learning Program Quality Assessment (Summer Learning PQA) (Smith, Helegda, et al., 2015).
What is available and how can I learn more?
You can download the Summer Learning PQA by clicking on the button below. In addition, a handbook to support the Summer Learning PQI process will be available for purchase in summer 2016. In general, it is likely that networks that already use either the Youth PQA or the CASP will find adapting to the new tool fairly straightforward. Networks that do not currently use either tool will require a larger investment in training staff and assessors. If you are interested in implementing the Summer Learning PQI or would simply like more information, you are encouraged to reach out to us to schedule a call.
February 4, 2016 | Webinar | Summer Learning Program Quality Intervention | Click here to view recording
November 24, 2014 | Webinar | Summer Learning Program Quality Intervention | Click here to view recording
May 25, 2016 | Webinar | Understanding the Summer Learning PQA| Click here to view the recording