A New Vision for School Accountability

In 2015, the Every Student Succeeds Act, or ESSA, replaced the 2001 No Child Left Behind Act, or NCLB, as the nation’s major K-12 education law, continuing that law’s focus on increasing the quality of public education. To fulfill this mission, ESSA requires states to measure, report on, and improve public school performance. Given the 14-year gap between ESSA and NCLB, the ways in which the old law measured and improved school quality were no longer useful in improving student outcomes.¹ States began requesting exemptions from the law’s more punitive measures in 2011.²

NCLB relied heavily upon a pass/fail system to measure school performance based on targets for test scores and graduation rates. ESSA marks a significant shift away from NCLB in a number of areas but none more so than the requirements for how states must hold districts and schools accountable for improving student outcomes. In particular, there are three key shifts in the approach to accountability.

First, ESSA moves beyond NCLB’s focus on test scores and graduation rates to a broader view of student and school success by requiring additional indicators and emphasizing the importance of a more holistic approach to accountability. Second, ESSA distributes responsibility for improvement among states, districts, and schools rather than focusing entirely on school-level actions directed by the state. And third, ESSA provides more flexibility at the local level for school improvement, requiring evidence-based strategies rather than the specific interventions of private tutoring and school choice that were mandatory for all struggling schools under the NCLB’s school improvement grants program.³

The new law’s vision for accountability recognizes that states need to build a systemic approach to prepare all students for college and careers—and they must do so quickly. While states are required by law to fully implement their accountability systems in the 2017-18 school year, even more urgently, workforce needs are changing rapidly.⁴ According to a recent study by the Center on Education and the Workforce, 99 percent of all jobs created since the market crash of 2008 require at least some postsecondary training.⁵ Most students can no longer compete in the economy without advanced training beyond a high school education. Furthermore, while graduation rates on the whole are on the rise and more low-income students and students of color are attending college, their rates of high school and college completion lag behind national totals.⁶ If all children are to succeed in college and careers, then states must continue to tackle the persistent gaps in educational attainment for particular groups of students.

However, ESSA’s approach is incomplete. Systems built solely for ESSA compliance inform states, districts, schools, and the public of what outcomes students met without explaining why they met them. As a result, states have been building toward more comprehensive accountability systems in recent years. In a 2014 report on next-generation accountability systems, the Center for American Progress reviewed how states were expanding their accountability systems to better support school and district improvement.⁷ The report identified five broad categories into which states are organizing their reforms and used those categories to formulate a new concept for accountability. The categories are:

• Measuring progress toward college and career readiness
• Diagnosing and responding to challenges via school-based quality improvement
• State systems of support and intervention
• Resource accountability
• Professional accountability

Building off of that review, this report describes a comprehensive approach to school accountability that encompasses each of these categories and goes beyond ESSA’s vision to help states, districts, and schools understand what led to their results. The report reviews the ESSA accountability requirements; describes a broader vision for student and school success; details a system for process management that fosters systems-level accountability to help states understand how well they are progressing toward that broader vision; and provides considerations that states should keep in mind when building accountability systems.

The report’s school accountability approach emphasizes two equally important goals for these new systems: 1) ensuring that accountability systems drive toward equal education opportunities by creating a system for identifying and acting on chronic low performance by particular groups of students and 2) ensuring that accountability systems are broadly framed in order to drive toward a comprehensive conception of student and school success and a culture of continuous improvement rather than just shame and punishment.

In order to achieve these goals, CAP proposes that states think holistically when choosing the data used to measure student, school, and district success, as well as consider carefully how data are used. We suggest that states build two connected components for their system. One component is an ESSA-required system that leads to actions to improve school quality, and the other component is a system that helps states understand what led to those outcomes.

This approach follows CAP’s belief—laid out in its 2014 report—that an ideal accountability system is meaningful for all schools when it embeds what ESSA requires within a broader system for driving improvements and supports. This includes a broad set of measures for student success; attention to district-level, not just school-level, accountability; development of systems for supporting schools and districts; improvements in how teachers are trained and supported; and accountability for how resources are allocated.

The Every Student Succeeds Act describes a continuous cycle of reporting student data, issuing school classifications, and using collected data to inform local interventions and supports. (see Figure 1)

As depicted in Figure 1, each of the activities that states undertake play a critical role in an accountability system. Furthermore, there are specific requirements for each of these activities within ESSA.

Student outcome data provides the bedrock for the entire system. ESSA requires that these data drive school classifications and school improvement efforts. In addition, these data provide transparency—or an honest accounting of how well students are doing. To serve each of these purposes, states will use a wider range of long- and short-term outcomes, as well as contextual data about the conditions of learning present in schools and districts, than was required by NCLB. What follows is a brief description of ESSA’s requirements for what must be reported and how schools are to be classified and improved.

Data reporting

States must annually and publicly report on how well all of their public school students are performing on the following measures, as well as set goals for the first, second, and fourth indicators.

1. Academic achievement in reading and math for third grade through eighth grade and once in high school
2. High school graduation rate
3. Growth or another academic indicator for elementary and middle schools
4. English language proficiency for English learners only
5. At least one measure of school quality or student success

Additionally, states must collect and report on a new, more varied set of data than the five indicators listed above. These new data provide insights into levels of student engagement and the availability of resources that support broader student learning. These data include access to advanced coursework, exclusionary discipline rates, chronic absenteeism, professional qualifications of educators, per-pupil expenditures, and postsecondary enrollment rates.⁸

School classifications

States must use the five indicators listed above for the 2017-18 school year, and every three years thereafter, to identify a subset of their lowest-performing schools according to the performance goals that states set for the specific indicators listed above.

Collectively, there are five types of low-performing schools, including those receiving ESSA funds under Title I of the law as well as any public schools meeting the criteria listed below.⁹

Comprehensive support and improvement schools, identified once every three years

• Lowest-performing: Lowest-performing 5 percent of schools in the state participating in Title I
• Low graduation rate: Any public high school with graduation rates less than 67 percent
• Chronically low-performing subgroup: Any Title I school previously identified for targeted support and improvement that fails to meet the state’s exit criteria after implementing interventions

Targeted support and improvement

• Consistently underperforming subgroup: Any school with one or more consistently underperforming subgroups, identified annually
• Low-performing subgroup: Any school with one or more subgroups performing at or below the rate of all students in a school that is in the bottom 5 percent of schools statewide; any Title I schools so identified become “chronically low-performing schools” after failing to meet exit criteria¹⁰

School improvement

Identified schools must implement evidence-based interventions and supports until they meet state-set exit criteria, and districts must support these schools in selecting and implementing the interventions and supports that fit the schools’ identified needs. Districts with several identified schools must review resource allocation and address it in the schools’ improvement plans. States must also identify additional actions for schools that fail to meet state-set exit criteria.

Because ESSA’s school quality and improvement requirements are limited to the above measures, states should consider what a broader vision for school and student success looks like.

College and career readiness is a central policy goal of the Every Student Succeeds Act and a running theme of the law, even though the legislation falls short of defining or even mentioning the term outright. To varying degrees, the requirements for key provisions of the law—including standards, assessments, accountability, school improvement, other student supports, and educator effectiveness—speak to the need to prepare students for advanced training after high school and specifically call for students to receive a “well-rounded education.”¹¹ As a result, the law acts as a broad framework for college and career readiness that states can further define through their implementation of the law. At the same time, states were working to define college and career readiness for several years prior to ESSA being passed.

Definitions of college and career readiness are formal and informal statements on what range of academic knowledge and cognitive and practical strategies states believe that their systems of education should provide to students so that they are successful in college, the workforce, and society. Formal definitions have been codified in various ways, whether through state laws, regulations, or other major policy documents. Informal definitions are not codified specifically but are recorded in documents such as applications for federal funding.

In its review of both formal and informal state definitions of college and career readiness, a 2014 report by the College and Career Readiness and Success Center notes that state definitions mention the following, to varying degree and frequency:¹²

• Mastery of core academic content knowledge, including math, reading, writing, science, social studies, and history
• Attainment of skills related to critical thinking and problem-solving
• Skills related to social emotional learning, collaboration, and communication
• Civic and community engagement skills

Regardless of their formality, definitions of college and career readiness can be important drivers of state-level policy. By providing a common understanding of the term, they can promote coherence among the policies and strategies that relate to K-12 education but that reside outside its explicit scope. These policy areas may include but are not limited to a state’s health and development, social services, early learning, higher education, and workforce systems.

Furthermore, definitions help these state-level systems determine the most appropriate ways to measure college and career readiness from their own unique perspectives and authority. For example, there are important benchmarks within parent-child interactions in children’s early years that affect their lifelong ability to learn. A 1995 study from the University of Kansas showed that higher-income children were exposed to 30 million more words than children from low-income homes. The study also showed later deficits in learning associated with this gap.¹³ Given this reality, vocabulary attainment in the early years might be an important measure to collect for both health and early learning systems.

State definitions of college and career readiness can also promote coherence within the K-12 education system, specifically as states develop plans and strategies to implement ESSA. The most obvious example of this is through states’ adoption and implementation of academic standards and assessments that align with the knowledge and skills students need to enter credit-bearing coursework in college. To be sure, state adoption of college- and career-readiness standards—such as the Common Core State Standards, as well as their aligned, high-quality assessments—meet this requirement.¹⁴

Furthermore, states can also address college and career readiness in how they measure and classify school performance. For example, there might be age- or grade-band specific benchmarks that schools and districts should pay attention to, such as vocabulary attainment, and states can collect and report this information. States may not wish to use the entire range of knowledge, skills, and experiences related to college and career readiness in the measures they use to classify school performance, but much of this information can be useful to inform local educational practice within districts, schools, and classrooms.

At the same time, states are already using a number of college- and career-readiness indicators in their school classification systems. Most of these apply to high schools, making the case for learning more about which earlier college- and career-readiness benchmarks are important to track for a student’s earlier education and development. For more information on what college- and career-readiness indicators states are currently using to classify school performance, see CAP’s “Making the Grade” report.¹⁵

See Also

Making the Grade

ReportMay 19, 2016

Making the Grade

May 19, 2016

Carmel Martin, Scott Sargrad, Samantha Batel

To create an accountability system that explains not just what outcomes were reached but what decisions led to those outcomes, states should consider measuring the effectiveness of coordination among and between each level of the system: states, districts, and schools.

Designing and measuring effective coordination and interaction between states and districts

Clearly distinguishing who is responsible for ensuring that students are college and career ready; what they are responsible for; and how they are responsible helps each level of the system—states, districts, and schools—use their limited resources to reach a commonly understood goal for student and school success. Likewise, states can support more effective interaction within and between each level of the system when they know who does what in order to ensure effective leveraging of the tools and resources that the state provides.

Any highly functioning system continually audits its resources and reassesses how to allocate them to meet its goals. The same is true for systems of education.

One desired outcome of K-12 education is college and career readiness for all students. However, states’ short-term goals—or outputs—for college and career readiness should differ by school and context. Some schools may need more aggressive targets for student growth or for improving how safe and nurtured students feel on campus—commonly referred to as a school’s climate—than other schools. On the other hand, the baseline expectations for long-term outcomes should be the same for all schools. This means that over time, all schools should be expected to meet the same long-term targets for proficiency and graduation.

In designing systems of healthy interaction within and between states, districts, and schools, the critical questions states must ask are: What are the reasonable, short-term outcomes that states, districts, and schools can expect? How are these measured and by whom and how often are the results reviewed? States’ answers to these questions should inform their development of the metrics, benchmarks, and processes foundational to their accountability systems.

As Figure 2 shows, there should be a direct relationship among inputs, processes, outputs, and outcomes within state, district, and school systems, as well as between each of them. For example, a state’s academic standards ought to determine the type of curriculum—inputs—and teacher training—processes—that districts provide. Table 1 below shows how states can organize a system of inputs, processes, outputs, and outcomes. This organization is meant to be illustrative, not exhaustive.

States have been working toward building comprehensive, next-generation accountability systems that are made up of multiple components, including:

• Measuring progress toward college and career readiness
• Diagnosing and responding to challenges via school-based quality improvement
• State systems of support and intervention
• Resource accountability
• Professional accountability

Understanding how all of these components fit together within a system of inputs, processes, outputs, and outcomes can bring greater clarity to how to operationalize these systems cohesively.

The matrix detailed in Table 1 below shows how such a system can be organized. Note that the list included in Table 1 is not comprehensive enough to represent the entire scope of state work within accountability, but it is a start. States may wish to list additional items on this list that further capture the breadth of their work.

The inputs below are important foundational components of the public education system. It is essential that states effectively manage these inputs internally and deliver them successfully to districts. To do so, states must have the capacity to build and maintain high-quality inputs and effective processes in each of the categories of accountability. A first step toward building this capacity should include an assessment of the current status of inputs and processes, measured against the goal of college and career readiness. For example, states may wish to review the extent to which course curricula reinforce the state’s college- and career-readiness standards or conduct a similar review with respect to the state’s educator standards and licensure requirements. The goal of this review would be to measure the extent to which a state’s inputs and processes will result in the attainment of college and career readiness for all students. The following sections detail how each level of the system can have the greatest effect on student outcomes.

State actions

State governments are far removed from classrooms. Still, state-level processes can and do affect student outcomes. For example, academic standards are a key state-level input that has a major impact at the local level. A critical component of the academic standards adoption life cycle, which includes the development, review, and adoption of academic standards, is also the implementation of those standards. To be effective in teaching students to meet the adopted standards, teachers must receive information about what technical and instructional shifts are necessary to ensure student achievement. While teacher preparation and professional development are not generally thought of as accountability indicators, students are not likely to achieve the standards if teachers and leaders are not adequately prepared to teach them. Therefore, measures of teacher practice can be an important set of metrics for states to collect and review continuously.

In addition to these state-specific functions, states must identify and respond to low capacity and performance at the district level. While it is important that states pay attention to all district practices for teaching and learning, ESSA speaks to some very specific district-level capacities that states must monitor. In particular, these include the capacity of districts to implement evidence-based reforms in schools identified as low performing, as well as their capacity to monitor the distribution of resources when districts have a preponderance of these schools within their districts.

District actions

In many ways, district-level processes can have the greatest impact on student outputs and outcomes. For example, among all in-school factors, research has shown teachers to have the greatest impact on student achievement.¹⁷ Additionally, low-income students and students of color are disproportionately taught by inexperienced, unqualified, or out-of-field teachers.¹⁸ Districts, not states, control hiring, placement, and professional development processes. Therefore, accountability systems should measure district-level outputs such as equitable distribution of effective teachers and mastery of instructional practice.

In addition, since decisions about the distribution of resources to schools occurs primarily, though not entirely, at the district level, accountability systems should include measures of district-level resource distribution and how well-aligned resources are to student needs.

Districts must also monitor school-level capacity to carry out school functions. Although this monitoring should be broad in scope when capturing matters of teaching and learning, districts must pay special attention to schools identified as low performing to ensure that they carry out implementation of school improvement efforts effectively. Additionally, districts may also want to closely monitor schools not identified for improvement but whose performance indicates that the school is struggling. Paying sufficient attention to schools that are doing well overall is another important function of districts and part of the system of continuous improvement. Understanding the strategies for continuous improvement of schools not identified for improvement is a less understood topic; as a result, CAP is considering developing a resource that describes state and district approaches to supporting these schools.

School actions

Many of the conditions governing school decision-making are beyond the control of school teachers and leaders. However, there are critical areas in which principals and teachers have significant authority to make important changes that positively affect students. For example, there is significant evidence demonstrating that both lowering the rate of expulsion among students of color and establishing a culture of high expectations signaling that all children can and should excel often lead to higher student achievement and graduation rates.¹⁹

Unlike districts or states, schools are best positioned to establish a positive, inclusive, safe, and nurturing culture and climate. How well school leaders assess personnel and student needs around safety, inclusivity, and high expectations is an important set of metrics to include in an accountability system.

Figure 3 shows a flow chart of inputs, processes, outputs, and outcomes in an accountability system. If both inputs and the processes through which they are used are high quality, states can expect to see positive student outcomes. Without high-quality inputs and processes, any positive short- and long-term outcomes will happen sporadically and in spite of the accountability system—not because of it.

Including specific metrics that assess the inputs and outputs of state, district, and school actions is critical to understanding the reasons for short- and long-term outcomes. The next section explores how that information should be collected and reported to ensure that each level’s actions are coordinated.

As states consider how to design their processes, they should keep the following considerations in mind.

Anchor the system with goals and expectations

College and career readiness is one desired outcome of the K-12 education system. Defining college and career readiness with a level of specificity makes it easier to identify which inputs and processes at the state, district, and school levels contribute to achieving this goal. The state should also consider articulating additional goals for the system, such as preparing graduates to be effective participants in our democratic government.

In addition, an explicit and agreed-upon definition of college and career readiness is a powerful tool to create cohesion not just within the K-12 education system but also between the other systems that support long-term student success. For example, a state definition of college and career readiness also makes it easier for the state K-12 system to engage with the labor and higher education systems to create a more efficient network of college and career pathways.

In addition to student outcomes, there may be other critical goals for a school accountability system to achieve, such as fair and effective distribution of inputs. States could also set goals for the delivery and distribution of resources to districts and schools.

Clearly stating the goals for the system and aligning accountability metrics to those goals creates an important north star for which all actors within the system should aim.

Use data to monitor the health of the system

Tracking inputs and processes can provide states with essential contextual information. This information can give states critical intelligence that can help them anticipate or diagnose problems and facilitate problem-solving.

States may wish to understand how well-aligned inputs are to student needs and how inputs are used by districts and schools to address student needs. This type of data can also be critical to collect and review at the district and school levels.

However, in order for educators and policymakers to be candid about how well a system of inputs and processes is working, states ought to think carefully about what, if any, stakes are attached to the results. How that information is acted upon should foster a spirit of continuous improvement.

Define what quality inputs and processes look like

Low-quality inputs will likely result in low-quality outcomes. Therefore, states ought to spend time defining what high-quality inputs look like. For example, states may define a high-quality data system as one that tracks K-12, postsecondary, and workforce outcomes for all students.

Likewise, low-quality processes will likely lead to low-quality outcomes. It may be useful for states to describe the elements, listed below, of a high-quality process according to the literature on general process design:²⁰

• Consistency: States communicate to districts and school exactly what to expect in a timely manner.
• Quality: Inputs and processes meet the needs of districts and schools.
• Efficiency: Processes aim to minimize cost.
• Effectiveness: Processes satisfy the goal of college and career readiness for all students.²¹

Developing this level of clarity ought to be a collaborative effort among states, districts, and schools, as the latter two can provide critical local, contextual information not readily available to states about the types of inputs and processes that meet local needs. When inputs or processes fail to meet standards of high quality, states, districts, and schools can course correct.

Each of these considerations applies equally to the horizontal and vertical relationships between the inputs, processes, outputs, and outcomes.

Another factor critical in the design of an accountability system is how state-level indicators identify schools most needing support, as well as drive behavior at the district and school levels.

Considerations for indicators that classify school performance

Regular measurement and reporting keeps schools and districts focused on what actions to take to improve performance. As described in the overview of the Every Student Succeeds Act, classifying school performance carries specific and enhanced consequences for schools identified as low performing. These schools must implement evidence-based interventions and must exit low-performing status within state-set timelines. Given this reality, states should take special care when selecting indicators that they will use to classify schools. ESSA requires indicators to be valid—or measure what they purport to measure; reliable—measure a specific result consistently over time; and comparable—measure the same element of performance across different schools.²²

When selecting indicators to classify schools into categories, states should also examine three additional characteristics for each indicator: differentiation between schools, relationship to key student outcomes, and ability to drive behavior. Based on these characteristics, states can then determine the most appropriate way to use them in the system—for example, in classification of schools, public reporting, or needs assessment and improvement planning—as well as the appropriate level—state, district, or school—at which to use them.

States could consider indicators that do not meet these three characteristics or that are otherwise not technically valid, reliable, and comparable across schools for other purposes in their accountability system but should not use them to classify schools.

Meaningful differentiation of school quality and performance

Generally, indicators used to classify schools for intervention purposes should distinguish performance between schools. Meaningful differentiation helps states prioritize which schools need the most support in improving and helps parents understand how their children’s school measures up to others.²³

As a result, states should analyze whether similar types of indicators differentiate more effectively than others. For example, if schools cluster around a value or range of values on a particular indicator, this indicator may not provide useful information to distinguish school performance. Meaningful differentiation would likely show performance across a range of values, showing performance at the bottom, middle, and top of the performance spectrum.

Historically, indicators including academic proficiency rates and graduation rates have widely varying performance from school to school, while indicators such as attendance rates typically have the same performance across all schools. In contrast, looking at chronic absenteeism would likely identify outliers in terms of performance. Also, while states are required to measure academic proficiency indicators—which are static, point-in-time indicators within their school classification systems—there may be an opportunity to measure specific aspects of proficiency data, such as growth or scale scores, which are further described below.²⁴ That is, ESSA may provide states an opportunity to use differentiation within an indicator, as well as differentiation between schools.

While differentiating between school performance is important, indicators that do not differentiate well might still be useful for school classification if they send critical signals about what is important and what schools should focus on. For example, nearly every school has high attendance rates, so this indicator does not differentiate among school performance. However, states may still be interested in attendance data and may wish to measure rates of chronic absenteeism, or the number of students who miss 10 or more days of school in a year. Schools would perform well on this measure if they reduce rates of chronic absenteeism, and states could focus their attention on schools not reducing these rates.

For indicators used to inform decision-making at the district or school level, and not used to classify schools at the state level, differentiation of schools is less important. What matters more to districts and schools is that teachers and school leaders can act on the data.

Actionable data may not meet the technical standards required by ESSA but are useful in informing practice. Examples of this type of data include measures of social and emotional learning, or SEL, which are the skills and abilities that provide a foundation for lifelong learning and development. The Collaborative for Academic, Social, and Emotional Learning describes five core competencies that make up SEL. These are self-awareness, self-management, social awareness, relationship skills, and responsible decision-making.²⁶ While there are existing assessments that measure aspects of SEL for particular groups of students, none of these assessments’ purpose is to hold schools accountable for students’ SEL.²⁷ Nonetheless, providing local educators with insights into how well students are developing their SEL can be useful to inform instructional practice.

Relationship to key student outcome measures

The flexibility to include nonacademic indicators, such as chronic absenteeism in school classification systems, provides an opportunity for states to identify indicators that provide unique and useful information about a school’s performance and key student outcomes. For example, states should include indicators that have a strong correlation with particular outcomes—including proficiency or graduation rates—but including too many of these can be redundant. On the other hand, another indicator might be so weakly correlated that it may have little or no relationship to critical student outcomes. As a result, states would not want schools to focus on this indicator. Ideally, indicators used for classification purposes would have a moderate to strong relationship with key student outcome measures, particularly long-term outcome measures such as college completion rates.

However, some indicators have weaker relationships with long-term outcomes but still provide useful information at the district and school levels that local educators can act upon. Indicators are particularly actionable when they can inform real-time decision-making for district or school resource allocation or another aspect of educational practice. Growth data that comes from assessments administered during the school year, for example, can help educators adjust their instructional practice throughout the year.

Indicators that drive behavior

Indicators used to classify schools should drive the kind of behavior that states want to see at the district and school levels. That is, these indicators should support schools in taking actions focused on the advancement of a state’s goals.

An indicator may not have strong relationships with student outcomes or provide meaningful differentiation between school performance but still provide value in a school classification system, particularly if that indicator signals what a state values and drives behavior that states want to see at the district and school levels. For example, parent engagement, as measured by survey responses, may not be correlated with student outcomes or differentiate among school performance but is an activity that the state wishes that schools would emphasize.

In this case, it will be important for a state to describe and be transparent about what value the indicator represents in cases where that is unclear and what action districts and schools should take based on school performance on that indicator.

Some indicators may not have all three characteristics shown in Figure 4, above. To maximize the value that they provide to policymakers, administrators, and educators, however, indicators used to classify school performance would ideally share all three. As a result, when making decisions about what indicators to use for school classification, states should analyze the extent to which possible indicators have one or more of these characteristics.

Accountability systems should drive continuous improvement toward making college and career readiness a reality for all students. To do so, states must build accountability systems that exceed ESSA’s requirements and focus on coherence within the entire system—from schools to districts to state educational agencies. CAP’s accountability framework calls for states to continuously monitor resources provided to districts and schools and ensure that they have the capacity to use them effectively. As a result, states will set goals and monitor progress against key functions such as training and support of teachers, as well as the distribution of financial and material resources. Finally, states should think through their mechanisms to support districts to use all of these resources effectively, while districts should pay attention to building school capacity.

Laura Jimenez is the Director of Standards and Accountability for the K-12 Education Policy team at the Center for American Progress.

Scott Sargrad is the Managing Director for the K-12 Education Policy team at the Center.