Productive Connections (DST 2011-12)

Executive Summary

Background

As one key component of North Carolina’s 4-year, $400 million Race to the Top (RttT) grant
activities, the District and School Transformation Division (DST) of the NC Department of
Public Instruction (NCDPI) is intervening to improve student achievement and high school
graduation rates in the lowest-achieving five percent of the state’s high schools, middle schools,
and elementary schools, some 118 schools in all. In addition, the DST is working with twelve of
the lowest-achieving school districts in the state to strengthen their ability to lead and support
effective school reform. As expressed in the current scope of work for the RttT grant, the goals
of the initiative are these:

  1. Turn around the lowest 5% of conventional elementary, middle, and high schools based on
    the 2009-10 Performance Composite and grade span,
  2. Turn around conventional high schools with a 4-year cohort graduation rate below 60% in
    2009-10 and either 2008-09 or 2007-08, and
  3. Turn around the lowest-achieving districts with a 2009-10 Local Education Agency (LEA)
    performance Composite below 65%.

Three organizations—the Carolina Institute for Public Policy (UNC-Chapel Hill), the Friday
Institute for Educational Innovation (NC State University), and SERVE (UNC-Greensboro)—
have formed a consortium to evaluate the Race to the Top grant. The primary purpose of the
evaluation is to provide objective research to help the NCDPI adjust RttT work as it progresses, a
type of evaluation often labeled “formative” because it seeks mainly to help the client
organization shape work in progress rather than simply to render an up-or-down “summative”
judgment on its impact.

The present report is the second of four reports focusing on the work of the DST. The first
examined the division’s pre-Race to the Top interventions in low-achieving schools in an effort
to distill lessons from that earlier work that could guide interventions during the Race to the Top
grant period. In that study, we found that improvement had taken place through a process we
called scaffolded craftsmanship. The scaffolding consisted of a planning framework, professional
development, and coaching provided by the NCDPI and its partner organizations. With these
supports, school leaders and staff gradually learned how to improve performance through guided
reconstruction of key school functions rather than implementation of externally designed models.
The present report shifts the focus from school-level interventions to the DST’s district-level
work in the twelve lowest-achieving school districts in the state. In this round of study, we found
that in low-achieving districts, connections are weak or missing between and within levels of the
systems—the central office, schools, and classrooms. The essence of what the DST is doing is to
strengthen or create productive connections across and between levels of the systems. Before explaining more fully what we mean by this term, we pause briefly to outline our methods and
purpose.

Study Methods and Purpose

By conducting interviews with DST coaches and local educators in four districts that are at
different points in the transformation process, supplemented by review of documents such as
strategic plans and needs assessments, we sought to identify the essential elements of the district
transformation process and to develop a rough “theory” of how that process takes place. That is,
we attempted not only to describe the steps taken by the DST and local educators, but to
highlight the essentials of that process, including the factors that impede as well as those which
facilitate progress. By isolating the essential components of the process, we sought to develop a
clear and concise account of the dynamics of district transformation. In this report, we offer a
relatively detailed account of the process, but we also present a more theoretical account—a
series of propositions or hypotheses that sum up the process in an economical way. This
theoretical account is intended to help DST leaders and coaches grasp the essentials of the
transformation process in a way that may be difficult in the midst of the very complex change
process they are engaged in. It is, however, just a “first draft” of a theory that we will test and
refine through additional study over the next two years.

Findings

The theory that we have derived from the interview and other data we collected in the four
districts in our sample can be summarized in the following propositions:

  • In low-achieving districts, a first challenge is to establish the improvement of student
    achievement and related student outcomes as the central goal of the school board and
    superintendent, not just as a broad policy, but in the continuing flow of specific decisions that
    arise over time. The proliferation of plans based on mandates or requirements imposed from
    many sources and the potentially conflicting claims of multiple community constituencies
    pose ongoing threats to the preservation of a dominant focus on student achievement.
  • Further, in low-achieving districts, connections are missing or weakly developed at many
    junctures up and down the system. That is, many junctures across and within the levels of a
    district lack one or more of the elements of a productive connection. Productive connections
    include:
  1. the combination of assertive accountability and bonds of relational trust and engagement
    that fosters commitment to improve student achievement,
  2. the provision of guidance, instruction, and assistance that builds the knowledge and skills
    necessary to improve performance, and
  3. the ongoing support for and monitoring of good practice, assessment of outcomes, and
    use of assessment results to improve practice which assure that commitment, knowledge,
    and skills are actually put into practice to produce the desired outcomes.
  • By “junctures” we mean the connections between superintendents and their boards; between
    superintendents and central office administrators; among central office administrators;
    between superintendents and central office administrators on the one hand and principals on the other; among principals across schools; between principals and teachers within schools,
    among teachers within schools or departments; between principals and teachers on the one
    hand and parents on the other; and between teachers and students.
  • The DST’s district level interventions are essentially efforts to strengthen or create
    productive connections at all of these junctures, thus weaving a web of support for the
    improvement of student achievement.
  • The elements of productive connections are similar across all of these junctures. In slightly
    different forms, they all involve (1) the combination of accountability and trusting
    relationships, (2) guidance, instruction, and assistance; and (3) monitoring practice, assessing
    outcomes, and using assessment results to improve practice.
  • To carry out the latter two functions—(2) to guide, instruct, assist, and (3) to monitor and
    improve practice — at any level of the system, leaders at each juncture need a clear,
    explicitly-defined concept of good practice. For example, a principal needs a well-defined
    image of good teaching as a basis for monitoring and shaping classroom instruction.
    Similarly, central administrators responsible for supervising principals need a well-defined
    image of good principal leadership.
  • To improve practice over time, leaders at each juncture also need an effective assessment
    system and knowledge of how to use assessment results to make changes in the shared image
    of good practice and in actual practice.
  • The more complete the web of productive connections in a district, the more student
    achievement will rise over time.
  • Pockets of poor achievement—such as a low-performing school or department—indicate
    failures to complete the web of productive connections.
  • A complete web of productive connections includes both links in the administrative chain of
    command between levels of the system and links among colleagues within levels of the
    system, the latter often referred to as professional communities. Absent productive
    professional links, productive administrative links will not be adequate to raise student
    achievement sharply.
  • The key capacity of an individual at any level of the system is the capacity to make
    productive connections, both with the people s/he is responsible for leading and with
    colleagues.
  • Professional development and coaching that are well-calibrated to the level of trust in the
    coach-client relationship and that attend to all elements of productive connections up and
    down the system can make strong contributions to the improvement of student achievement,
    but where connections remain weak after sustained intervention, personnel replacement is
    required.

In sum, “district transformation” is essentially the process of changing a disconnected district
into a productively connected district.

Conclusion

It would be premature to make any summative judgment of the degree to which the District and
School Transformation division has succeeded in transforming the districts where it is
intervening, but it may be useful to offer an interim assessment of progress to date. In schools
served by the DST, the two-year improvement in Performance Composites from the 2009-10 to
the 2011-12 school year clearly outpaced the statewide average improvement—by 8.8 percentage
points at the high school level, 4.7 points at the middle school level, and 7.1 points at the
elementary school level. Further, the improvement in Performance Composites among schools
where the DST was intervening at the district level in addition to the school level outpaced the
improvement in schools where the DST was intervening solely at the school level—by 13.2
percentage points at the high school level, 3.5 points at the middle school level, and 2 points at
the elementary school level. These findings suggest that the DST’s school-level interventions are
making a notable difference in performance improvement, and that the district-level
interventions are adding additional value beyond the school-level interventions by themselves.

In addition to improving student achievement as measured by Performance Composites, the DST
also set the goal of improving high school graduation rates. On this goal, the evidence is
encouraging for DST interventions overall, but offers less support for a unique contribution for
the district-level interventions. Statewide, from 2009-10 to 2011-12, high school graduation rates
improved by 6.2 percentage points. In high schools served by the DST, the average two-year
improvement was 9.5 percentage points—3.3 points more than in the state as a whole. This
suggests that the DST has contributed to improvement in the graduation rate for the schools it
served. But average graduation rates in high schools served solely through DST intervention at
the school level actually improved 1.2 points more than did high schools where the DST was
also intervening at the district level. The latter finding is not entirely surprising. The schools in
districts where DST chose to intervene started with what amounts to a double disadvantage—
they were low performing as schools but were also situated in low-performing districts. So it
may take longer to make a difference in these schools than in those located outside of lowachieving
districts.

All in all, it appears that the DST is making a measurable contribution to the improvement of
both performance and graduation rates in the schools it serves. Our findings from two rounds of
study suggest that the school-level improvements take place through a process of scaffolded
craftsmanship
and that the additional contributions of the district level interventions may result
from making productive connections up and down the school systems, thus supporting
scaffolded craftsmanship in the initially low-achieving schools in those districts. During the
remaining two years of our evaluation of the Race to the Top-supported efforts of the District
and School Transformation unit, we will examine these processes more fully in order to refine
our findings and test their validity.

View Resource