Overview of the Evaluation and Progress Made since the Last Report
This second annual report of the RttT STEM implementation activities documents ongoing
implementation of RttT STEM initiative in participating schools and assesses intermediate
outcomes for students and staff in anchor schools after one year of implementation.
The evaluation is guided by the following two research questions:
To what extent have the four key elements of the network of STEM anchor and affinity
schools (network structure, professional development, curriculum, and partnerships)
been implemented as intended?
What are the intermediate outcomes for students and staff in anchor schools after one
year of implementation?
In addition, this report notes recommendations from the Year 1 evaluation report that were
addressed during the second year of implementation of the initiative. In particular, the
Integrated the six North Carolina New Schools Project (NCNSP) Design Principles with the
various components of the STEM vision;
- Provided explicit training for leadership teams on creating a common STEM vision for staff;
Continued using the four NC Learning Lab Schools as sites for study visits by teams from
network schools while eventual anchor schools continued to develop;
Provided opportunities for schools that joined the network late to catch up via provision of
PD necessary for successful implementation of the STEM model;
Provided more background knowledge to teachers about the STEM themes and the
engineering design process prior to their work on projects;
- Engaged instructional coaches in supporting the project work;
- Actively involved IHE and business partners in designing a project-based curriculum;
Contracted with highly-skilled teachers to develop model projects for each of the four
affinity networks; and
Designed and used a standardized participant evaluation form for evaluations of multiple PD
offerings from NCNSP.
Findings and Recommendations
One of the initiative’s objectives was to “Work with partners to support the development of a
small set of anchor/model STEM high schools that will serve as laboratory schools and sites for
professional development around project-based learning.” There is definite progress toward this
goal, with three of the anchor schools working hard to improve instruction and implement STEM
features such as project-based learning, their STEM theme, and additional STEM courses, and
also utilizing partnerships for improvement of student learning. The fourth school is welcoming
their first students in the 2012–13 school year (with one-year delay). Based on analyses of RttT
STEM initiative activities to date, the Evaluation Team concluded that structures for networking,
professional development, curriculum development, and partnerships are in place to support both
anchor and affinity schools as intended, though some of these activities have been delayed. A
summary of findings and recommendations for each of the four areas of implementation
strategies and for the intermediate outcomes observed in the three anchor schools are presented
I. Structure of the Network of Stem Anchor and Affinity Schools
Baseline characteristics of the RttT-funded STEM schools
Prior to the initiative’s launch, RttT STEM schools offered a lower proportion of advanced
STEM courses than did the average high school in the state. In most cases, student
achievement in RttT STEM schools was not notably different from all other high schools;
however RttT STEM school physical science EOC scores did tend to be higher.
Since its launch, the RttT STEM initiative has made progress toward its goal of serving
minority and poor students, who are traditionally underrepresented in STEM fields. In 2010–
11, North Carolina RttT STEM Affinity Network schools served a higher proportion of black
and Hispanic students and a higher proportion of students of poverty than did the average
high school in the state, hosted the same proportion of female students, and were more likely
to be located in an urban area.
Also, while faculty credentials and experience were similar across RttT STEM Affinity
Network schools and all other high schools, per-pupil expenditures for STEM schools
typically were slightly higher on average, and school sizes often were smaller.
Face-to-face and online networking
NCNSP has encouraged and facilitated networking and collaboration by various means,
including embedding it in face-to-face PD events, furnishing online collaboration tools, and
providing coaching services. Currently, face-to-face meetings have been the most successful
Networking among schools in the STEM network is still in the early stages. Some schools
have been networking with other schools outside of the RttT network.
NCNSP provides multiple opportunities for online collaboration. Edmodo, the original
online network for STEM schools, has not been actively used.
Infrastructure developed for schools and their partners to share resources
As part of the RttT initiative, the NC STEM Learning Network was created and provided a
number of services and products, though some of the main products and services have not
been finished and require additional sustainable funds to continue in operation.
There has been little collaboration between the NC STEM Learning Network and the
NCNSP STEM network.
Leadership coaches should consider making increases in advanced math and science courses
a possible emphasis for conversations with administrative teams in RttT STEM schools.
Implementers should consider various strategies for increasing the appeal of and incentives
for visiting a virtual networking hub, including moving some PD elements for content and
instruction into the online space, and encouraging instructional and STEM coaches to create
online groups for following up on face-to-face visits.
In order to increase the effectiveness of sharing best STEM practices and resources, the
NCNSP Affinity Network and the North Carolina STEM Learning Network should consider
a better coordination of their activities. Additionally, creating a central hub (or portal), with
access to content resources, professional development, and assessment and lesson planning
tools that could serve both networks, might increase the utility and effectiveness of online
collaboration for both networks.
II. Professional Development
- Schools are receiving the PD and coaching services outlined in the scope of work.
Most of the coaching visits to comprehensive schools that joined the network in 2011
happened in 2012, and the number of visits per school was unevenly spread among schools.
Overall, PD and coaching were seen as valuable and of high quality. Staff at the anchor
schools hoped for continuing PD and coaching in the upcoming year.
Professional development was most appreciated when participants understood its direct
application to their classroom.
- The vast majority of coaching time was spent on changing instruction in the classrooms.
The fact that coaches engaged with schools over an extended period of time gave coaches,
teachers, and principals the opportunity to develop trusting relationships that likely increased
the coaches’ impact.
Challenges and barriers related to PD included:
- Sending teams to off-site PD during the school year for schools with small staffs;
- Balancing the competing demands of different RttT initiatives; and
- Getting buy-in from teachers around changing instruction.
Much of the professional development was perceived by recipients as relevant, but NCNSP
may want to explore ways of increasing the relevance of the lowest-rated sessions.
Because the impact of the coaches increased the longer they worked with teachers,
implementers should consider having coaches in larger schools focus initial efforts on
working intensively with a sub-set of teachers, instead of working with the entire faculty.
To better leverage professional development and coaching resources and to create incentives
for using online networking, the Implementation Team should consider blended professional
III. Development and Implementation of Project-Based Curricula
A new contract was awarded to the North Carolina School of Science and Mathematics
(NCSSM) by NCDPI to design STEM curricula with project units. Between July and August
2012, NCSSM delivered the outlines for all 16 year-long courses and the first units for the
four freshman courses in each of the four themes (Aerospace, Security and Automation,
Biotechnology and Agriscience, Energy and Sustainability, and Health and Life Sciences).
NCNSP provided multiple opportunities for teachers to engage in professional development
focused on the four themes and on project design and implementation.
Themes are being incorporated in anchor and affinity schools in a number of different ways,
including special sequences of courses on a theme, integrating a theme in all core subjects,
and blending two or more courses.
Three existing anchor schools started to incorporate both cross-curricular projects and
projects within individual subjects.
Scheduling and teacher knowledge on project-based learning (PBL) were identified as
challenges for project implementation.
The initiative leads should consider identifying additional resources and supplementary
funds to support piloting and revisions of and professional development for the 16 year-long
STEM courses. NCNSP should consider identifying schools from each of the Affinity
Networks that are willing to pilot the courses and provide feedback to the developers. In
addition, NCSSM should share the units with Affinity Network schools at scheduled
professional development events.
Based on teacher feedback, incorporation of themes and project design and implementation
should be emphasized both in professional development and in resource development efforts.
Based on principal feedback, the Implementation Team should consider providing schools
with tips and examples of schedules that allow for integration of themed and cross-curricular
projects in the context of a regular school day.
Industry Innovation Councils (IICs) for each of the four themes met quarterly to plan and
provide support for the networks.
Industry and IHE partners provided expertise to school staff on themes and on relevance to
local community economic development, and they also planned partnership activities with
NCNSP, with the help of business partners, is developing a sustainable and replicable
prototype model partnership to be implemented in the four themed networks.
Ongoing challenges for schools: building partnerships in rural areas; making partnerships
more collaborative and hands-on; and developing teacher content knowledge in the theme
and in teaching career-ready skills.
The model for partnership building is currently being developed in one of the urban schools;
the Implementation Team should consider examining specific issues faced by rural schools.
There are still a number of questions and issues related to partnerships that anchor schools
need to resolve, such as the anchor school’s role in providing partners to other schools in the
network, or in communicating between schools. The Implementation Team should devote
more time both face-to-face and online to the anchors or other groups of schools with
common issues and work together to resolve these issues.
V. Intermediate Outcomes for Students and Staff in Anchor Schools
- In all three anchor schools, the initiative remains in the beginning stages of implementation.
Given the large number of the early college/STEM design features that schools have to
implement, the anchor schools each start with different priorities, which are affected by their
context and by principals’ preferences.
- There is not yet universal buy-in into the STEM initiative among staff in the anchor schools.
All anchor schools added additional STEM courses, such as engineering, technology, science,
and health sciences; some schools are adopting more innovative math and science textbooks.
Technology is a high-priority area in all three schools, both as a subject of study and as an
instructional tool for learning content across subjects.
Many teachers report that they improved their instruction and implemented instructional
strategies emphasized by NCNSP professional development, such as collaboration,
classroom talk, inquiry and project-based learning, and higher order questioning.
Interviews with staff and students indicated that students in anchor schools enjoy
personalized attention and exhibit high motivation, engagement, and passion for learning.
Staff identified a number of challenges to overcome during implementation, such as better
defining and understanding the STEM model, improving teacher qualifications, increasing
student preparedness, and addressing logistical issues.
In acknowledgement of the struggles faced by many participating schools to define what this
initiative means for them and how to integrate multiple initiatives from the state, district, and
NCNSP, the Implementation Team should consider providing more differentiated help to
schools by staggering emphasis on different Design Principles and STEM features,
depending on each school’s context.
To help schools faced with logistical issues related to their conversion or start-up, the
Implementation Team should create resources and an online blog or discussion devoted
specifically to those issues.
Implementing the STEM initiative’s more innovative components such as thematic and
cross-curricular projects requires that teachers gain substantial new knowledge about both
content and instructional strategies. The Implementation Team should consider
differentiating ways of providing professional development devoted to these issues.
Continue to track changes in the demographic, financial, and academic measures of RttT
STEM schools through the administrative data, identify the degree to which any changes are
related to efforts connected to RttT, and use this evidence to determine progress toward the
stated goals of the North Carolina RttT STEM initiative.
- Continue qualitative data collection and analyses.
Analyze responses to staff and student surveys that were collected in Spring and Fall of 2012
to provide baseline data.
- Provide a more detailed report about RttT-funded NCSSM curriculum development activities.