Race to the Top Evaluation: STEM Affinity Network—Second-Year Report

Executive Summary

Overview of the Evaluation and Progress Made since the Last Report

This second annual report of the RttT STEM implementation activities documents ongoing
implementation of RttT STEM initiative in participating schools and assesses intermediate
outcomes for students and staff in anchor schools after one year of implementation.

The evaluation is guided by the following two research questions:

  1. To what extent have the four key elements of the network of STEM anchor and affinity
    schools (network structure, professional development, curriculum, and partnerships)
    been implemented as intended?
  2. What are the intermediate outcomes for students and staff in anchor schools after one
    year of implementation?

In addition, this report notes recommendations from the Year 1 evaluation report that were
addressed during the second year of implementation of the initiative. In particular, the
implementation team:

  • Integrated the six North Carolina New Schools Project (NCNSP) Design Principles with the
    various components of the STEM vision;
  • Provided explicit training for leadership teams on creating a common STEM vision for staff;
  • Continued using the four NC Learning Lab Schools as sites for study visits by teams from
    network schools while eventual anchor schools continued to develop;
  • Provided opportunities for schools that joined the network late to catch up via provision of
    PD necessary for successful implementation of the STEM model;
  • Provided more background knowledge to teachers about the STEM themes and the
    engineering design process prior to their work on projects;
  • Engaged instructional coaches in supporting the project work;
  • Actively involved IHE and business partners in designing a project-based curriculum;
  • Contracted with highly-skilled teachers to develop model projects for each of the four
    affinity networks; and
  • Designed and used a standardized participant evaluation form for evaluations of multiple PD
    offerings from NCNSP.

Findings and Recommendations

One of the initiative’s objectives was to “Work with partners to support the development of a
small set of anchor/model STEM high schools that will serve as laboratory schools and sites for
professional development around project-based learning.” There is definite progress toward this
goal, with three of the anchor schools working hard to improve instruction and implement STEM
features such as project-based learning, their STEM theme, and additional STEM courses, and
also utilizing partnerships for improvement of student learning. The fourth school is welcoming
their first students in the 2012–13 school year (with one-year delay). Based on analyses of RttT
STEM initiative activities to date, the Evaluation Team concluded that structures for networking,
professional development, curriculum development, and partnerships are in place to support both
anchor and affinity schools as intended, though some of these activities have been delayed. A
summary of findings and recommendations for each of the four areas of implementation
strategies and for the intermediate outcomes observed in the three anchor schools are presented
here.

I. Structure of the Network of Stem Anchor and Affinity Schools


Baseline characteristics of the RttT-funded STEM schools

  • Prior to the initiative’s launch, RttT STEM schools offered a lower proportion of advanced
    STEM courses than did the average high school in the state. In most cases, student
    achievement in RttT STEM schools was not notably different from all other high schools;
    however RttT STEM school physical science EOC scores did tend to be higher.
  • Since its launch, the RttT STEM initiative has made progress toward its goal of serving
    minority and poor students, who are traditionally underrepresented in STEM fields. In 2010–
    11, North Carolina RttT STEM Affinity Network schools served a higher proportion of black
    and Hispanic students and a higher proportion of students of poverty than did the average
    high school in the state, hosted the same proportion of female students, and were more likely
    to be located in an urban area.
  • Also, while faculty credentials and experience were similar across RttT STEM Affinity
    Network schools and all other high schools, per-pupil expenditures for STEM schools
    typically were slightly higher on average, and school sizes often were smaller.


Face-to-face and online networking

  • NCNSP has encouraged and facilitated networking and collaboration by various means,
    including embedding it in face-to-face PD events, furnishing online collaboration tools, and
    providing coaching services. Currently, face-to-face meetings have been the most successful
    networking channels.
  • Networking among schools in the STEM network is still in the early stages. Some schools
    have been networking with other schools outside of the RttT network.
  • NCNSP provides multiple opportunities for online collaboration. Edmodo, the original
    online network for STEM schools, has not been actively used.


Infrastructure developed for schools and their partners to share resources

  • As part of the RttT initiative, the NC STEM Learning Network was created and provided a
    number of services and products, though some of the main products and services have not
    been finished and require additional sustainable funds to continue in operation.
  • There has been little collaboration between the NC STEM Learning Network and the
    NCNSP STEM network.

Recommendations:

  • Leadership coaches should consider making increases in advanced math and science courses
    a possible emphasis for conversations with administrative teams in RttT STEM schools.
  • Implementers should consider various strategies for increasing the appeal of and incentives
    for visiting a virtual networking hub, including moving some PD elements for content and
    instruction into the online space, and encouraging instructional and STEM coaches to create
    online groups for following up on face-to-face visits.
  • In order to increase the effectiveness of sharing best STEM practices and resources, the
    NCNSP Affinity Network and the North Carolina STEM Learning Network should consider
    a better coordination of their activities. Additionally, creating a central hub (or portal), with
    access to content resources, professional development, and assessment and lesson planning
    tools that could serve both networks, might increase the utility and effectiveness of online
    collaboration for both networks.

II. Professional Development

  • Schools are receiving the PD and coaching services outlined in the scope of work.
  • Most of the coaching visits to comprehensive schools that joined the network in 2011
    happened in 2012, and the number of visits per school was unevenly spread among schools.
  • Overall, PD and coaching were seen as valuable and of high quality. Staff at the anchor
    schools hoped for continuing PD and coaching in the upcoming year.
  • Professional development was most appreciated when participants understood its direct
    application to their classroom.
  • The vast majority of coaching time was spent on changing instruction in the classrooms.
  • The fact that coaches engaged with schools over an extended period of time gave coaches,
    teachers, and principals the opportunity to develop trusting relationships that likely increased
    the coaches’ impact.
  • Challenges and barriers related to PD included:

    • Sending teams to off-site PD during the school year for schools with small staffs;
    • Balancing the competing demands of different RttT initiatives; and
    • Getting buy-in from teachers around changing instruction.

Recommendations:

  • Much of the professional development was perceived by recipients as relevant, but NCNSP
    may want to explore ways of increasing the relevance of the lowest-rated sessions.
  • Because the impact of the coaches increased the longer they worked with teachers,
    implementers should consider having coaches in larger schools focus initial efforts on
    working intensively with a sub-set of teachers, instead of working with the entire faculty.
  • To better leverage professional development and coaching resources and to create incentives
    for using online networking, the Implementation Team should consider blended professional
    development.

III. Development and Implementation of Project-Based Curricula

  • A new contract was awarded to the North Carolina School of Science and Mathematics
    (NCSSM) by NCDPI to design STEM curricula with project units. Between July and August
    2012, NCSSM delivered the outlines for all 16 year-long courses and the first units for the
    four freshman courses in each of the four themes (Aerospace, Security and Automation,
    Biotechnology and Agriscience, Energy and Sustainability, and Health and Life Sciences).
  • NCNSP provided multiple opportunities for teachers to engage in professional development
    focused on the four themes and on project design and implementation.
  • Themes are being incorporated in anchor and affinity schools in a number of different ways,
    including special sequences of courses on a theme, integrating a theme in all core subjects,
    and blending two or more courses.
  • Three existing anchor schools started to incorporate both cross-curricular projects and
    projects within individual subjects.
  • Scheduling and teacher knowledge on project-based learning (PBL) were identified as
    challenges for project implementation.

Recommendations:

  • The initiative leads should consider identifying additional resources and supplementary
    funds to support piloting and revisions of and professional development for the 16 year-long
    STEM courses. NCNSP should consider identifying schools from each of the Affinity
    Networks that are willing to pilot the courses and provide feedback to the developers. In
    addition, NCSSM should share the units with Affinity Network schools at scheduled
    professional development events.
  • Based on teacher feedback, incorporation of themes and project design and implementation
    should be emphasized both in professional development and in resource development efforts.
  • Based on principal feedback, the Implementation Team should consider providing schools
    with tips and examples of schedules that allow for integration of themed and cross-curricular
    projects in the context of a regular school day.

IV. Partnerships

  • Industry Innovation Councils (IICs) for each of the four themes met quarterly to plan and
    provide support for the networks.
  • Industry and IHE partners provided expertise to school staff on themes and on relevance to
    local community economic development, and they also planned partnership activities with
    schools.
  • NCNSP, with the help of business partners, is developing a sustainable and replicable
    prototype model partnership to be implemented in the four themed networks.
  • Ongoing challenges for schools: building partnerships in rural areas; making partnerships
    more collaborative and hands-on; and developing teacher content knowledge in the theme
    and in teaching career-ready skills.

Recommendations:

  • The model for partnership building is currently being developed in one of the urban schools;
    the Implementation Team should consider examining specific issues faced by rural schools.
  • There are still a number of questions and issues related to partnerships that anchor schools
    need to resolve, such as the anchor school’s role in providing partners to other schools in the
    network, or in communicating between schools. The Implementation Team should devote
    more time both face-to-face and online to the anchors or other groups of schools with
    common issues and work together to resolve these issues.

V. Intermediate Outcomes for Students and Staff in Anchor Schools

  • In all three anchor schools, the initiative remains in the beginning stages of implementation.
  • Given the large number of the early college/STEM design features that schools have to
    implement, the anchor schools each start with different priorities, which are affected by their
    context and by principals’ preferences.
  • There is not yet universal buy-in into the STEM initiative among staff in the anchor schools.
  • All anchor schools added additional STEM courses, such as engineering, technology, science,
    and health sciences; some schools are adopting more innovative math and science textbooks.
  • Technology is a high-priority area in all three schools, both as a subject of study and as an
    instructional tool for learning content across subjects.
  • Many teachers report that they improved their instruction and implemented instructional
    strategies emphasized by NCNSP professional development, such as collaboration,
    classroom talk, inquiry and project-based learning, and higher order questioning.
  • Interviews with staff and students indicated that students in anchor schools enjoy
    personalized attention and exhibit high motivation, engagement, and passion for learning.
  • Staff identified a number of challenges to overcome during implementation, such as better
    defining and understanding the STEM model, improving teacher qualifications, increasing
    student preparedness, and addressing logistical issues.

Recommendations:

  • In acknowledgement of the struggles faced by many participating schools to define what this
    initiative means for them and how to integrate multiple initiatives from the state, district, and
    NCNSP, the Implementation Team should consider providing more differentiated help to
    schools by staggering emphasis on different Design Principles and STEM features,
    depending on each school’s context.
  • To help schools faced with logistical issues related to their conversion or start-up, the
    Implementation Team should create resources and an online blog or discussion devoted
    specifically to those issues.
  • Implementing the STEM initiative’s more innovative components such as thematic and
    cross-curricular projects requires that teachers gain substantial new knowledge about both
    content and instructional strategies. The Implementation Team should consider
    differentiating ways of providing professional development devoted to these issues.

Next Steps

  • Continue to track changes in the demographic, financial, and academic measures of RttT
    STEM schools through the administrative data, identify the degree to which any changes are
    related to efforts connected to RttT, and use this evidence to determine progress toward the
    stated goals of the North Carolina RttT STEM initiative.
  • Continue qualitative data collection and analyses.
  • Analyze responses to staff and student surveys that were collected in Spring and Fall of 2012
    to provide baseline data.
  • Provide a more detailed report about RttT-funded NCSSM curriculum development activities.

View Resource