Within a series of prior posts (see, for example, here and here), I have written about what the Every Student Succeeds Act (ESSA), passed in December of 2015, means for the U.S., or more specifically states’ school and teacher evaluation systems as per the federal government’s prior mandates requiring their use of growth and value-added models (VAMs).
Related, states were recently (this past May) required to submit to the federal government their revised school and teacher evaluation plans, post ESSA, given how they have changed, or not. While I have a doctoral student currently gathering updated teacher evaluation data, state-by-state, and our preliminary findings indicate that “things” have not (yet) changed much post ESSA, at least at the teacher level of focus in this study and except for in a few states (e.g., Connecticut, Oklahoma), states still have the liberties to change that which they do on both ends (i.e., school and teacher accountability).
Recently, a colleague recently shared with me a study titled “Next Generation Accountability: A Vision for School Improvement Under ESSA” that warrants coverage here, in hopes that states are still “out there” trying to reform their school and teacher evaluation systems, of course, for the better. While the document was drafted by folks coming from the aforementioned state of Oklahoma, who are also affiliated with the Learning Policy Institute, it is important to note that the document was also vetted by some “heavy hitters” in this line of research including, but not limited to, David C. Berliner (Arizona State University), Peter W. Cookson Jr. (American Institutes for Research (AIR)), Linda Darling-Hammond (Stanford University), and William A. Firestone (Rutgers University).
As per ESSA, states are to have increased opportunities “to develop innovative strategies for advancing equity, measuring success, and developing cycles of continuous improvement” while using “multiple measures to assess school and student performance” (p. iii). Likewise, the authors of this report state that “A broader spectrum of indicators,
going well beyond a summary of annual test performance, seems necessary to account transparently for performance and assign responsibility for improvement.”
Here are some of their more specific recommendations that I found of value for blog followers:
- The continued use of a single composite indicator to reduce and then sort teachers or schools by their overall effectiveness or performance (e.g., using teacher “effectiveness” categories or school A–F letter grades) is myopic, to say the least. This is because doing this (a) misses all that truly “matters,” including multidimensional concepts and (non)cognitive competencies we want students to know and to be able to do, not captured by large-scale tests; and (b) inhibits the usefulness of what may be informative, stand-alone data (i.e., as taken from “multiple measures” individually) once these data are reduced and then collapsed so that they can be used for hierarchical categorizations and rankings. This also (c) very much trivializes the multiple causes of low achievement, also of importance and in much greater need of attention.
- Accordingly, “Next Generation” accountability systems should include “a broad palette of functionally significant indicators to replace [such] single composite indicators [as this] will likely be regarded as informational rather than controlling, thereby motivating stakeholders to action” (p. ix). Stakeholders should be defined in the following terms…
- “Next Generation” accountability systems should incorporate principles of “shared accountability,” whereby educational responsibility and accountability should be “distributed across system components and not foisted upon any one group of actors or stakeholders” (p. ix). “[E]xerting pressure on stakeholders who do not have direct control over [complex educational] elements is inappropriate and worse, harmful” (p. ix). Accordingly, the goal of “shared accountability” is to “create an accountability environment in which all participants [including governmental organizations] recognize their obligations and commitments in relation to each other” (p. ix) and their collective educational goals.
- To facilitate this, “Next Generation” information systems should be designed and implemented in order to service the “dual reporting needs of compliance with federal mandates and the particular improvement needs of a state’s schools,” while also addressing “the different information needs of state, district, school site
leadership, teachers, and parents” (p. ix). Data may include, at minimum, data on school resources, processes, outcomes, and other nuanced indicators, and this information must be made transparent and accessible in order for all types of data users to be responsive, holistically and individually (e.g, at school or classroom levels). The formative functions of such “Next Generation” informational systems, accordingly, take priority, at least for initial terms, until informational data can be used to, with priority, “identify and transform schools in catastrophic failure” (p. ix).
- Related, all test- or other educational measurement-related components of states’ “Next Generation” statutes and policies should adhere to the Standards for Educational and Psychological Testing, and more specifically their definitions of reliability, validity, bias, fairness, and the like. Statutes and policies should also be written “in the least restrictive and prescriptive terms possible to allow for [continous] corrective action and improvement” (p. x).
- Finally, “Next Generation” accountability systems should adhere to the following five essentials: “(a) state, district, and school leaders must create a system-wide culture grounded in “learning to improve;” (b) learning to improve using [the aforementioned informational systems also] necessitates the [overall] development of [students’] strong pedagogical data-literacy skills; (c) resources in addition to funding—including time, access to expertise, and collaborative opportunities—should be prioritized for sustaining these ongoing improvement efforts; (d) there must be a coherent structure of state-level support for learning to improve, including the development of a strong Longitudinal Data System (LDS) infrastructure; and (e) educator labor market policy in some states may need adjustment to support the above elements” (p. x).
To read more, please access the full report here.
In sum, “Next Generation” accountability systems aim at “a loftier goal—universal college and career readiness—a goal that current accountability systems were not designed to achieve. To reach this higher level, next generation accountability must embrace a wider vision, distribute trustworthy performance information, and build support infrastructure, while eliciting the assent, support, and enthusiasm of citizens and educators” (p. vii).
As briefly noted prior, “a few states have been working to put more supportive, humane accountability systems in place, but others remain stuck in a compliance mindset that undermines their ability to design effective accountability systems” (p. vii). Perhaps (or perhaps likely) this is because for the past decade or so states invested so much time, effort, and money to “reforming” their prior teacher evaluations systems as formerly required by the federal government. This included investments in states’ growth models of VAMs, onto which many/most states seem to be holding firm.
Hence, while it seems that the residual effects of the federal governments’ former efforts are still dominating states’ actions with regards to educational accountability, hopefully some states can at least begin to lead the way to what will likely yield the educational reform…still desired…
Econometric thinking is driving educational policy and practice today.
I find this report free of any serious discussion of what education is for. In my judgment the report is too preoccupied with ESSA and assumptions about schools that pre-date the current and unfolding Trump/Devos regime.
What does accountability mean when public education, especially in place-based schools, is some remainder left when choice, markets, and so-called personalized delivery of education are held out as the future of education?
What does accountability mean when there is a collapse of belief in the value and necessity of education and the obligation of the nation, the state, and local community to provide for it/
This report assumes that the nation will continue to have a centralized and comprehensive accountability system, data-driven of course, with longitudinal data systems tweaked and massaged and presented in dashboards that provide guidance on what students and educators need to do.
In my opinion, the report is pre-occupied with questionable ideas. One is a version of continuous improvement (vintage 1970s) re-furbished and re-branded as a “science of learning to improve,” complete with a “known step-by-step procedure for addressing “complexity.” The report speaks of a post-industrial age and 21st Century skills as if these are irrefutably the most important conditions that bear on education for this generation. No attention to the emerging “minority majority” and what that means for an accountability system.
The report posits two purposes for education: college and career readiness and deeper learning. Here are the definitions.
(a) College and career readiness:“The preparation of high school graduates to enter a career, extended training, or specialized education without need for remediation. That language is straight from Achieve, Inc. and the promoters of the Common Core.
(b) Deeper learning: “The ability to transfer knowledge and skills developed in one setting to new situations, contexts, and problems through a set of cognitive, intrapersonal, and interpersonal competencies. “ The language of “competencies” is as troubling as the failure to acknowledge that the construct “deeper learning” has been pushed by the tech industry.
I think I am most troubled by the idea that sufficient data, especially longitudinal data will lead educators to wise, prudent, and just decisions. I think not. Consider the purposes of the Gates funded Teacher-Student Data Link Project and Data Quality Campaign:This campaign envisions the link between teacher and student data serving eight purposes:
1. Determine which teachers help students become college-ready and successful, 2. Determine characteristics of effective educators, 3. Identify programs that prepare highly qualified and effective teachers, 4. Assess the value of non-traditional teacher preparation programs, 5. Evaluate professional development programs, 6. Determine variables that help or hinder student learning, 7. Plan effective assistance for teachers early in their career, and 8. Inform policy makers of best value practices, including compensation (TSDL, 2011, “Use and Purpose”).
The system is intended to monitor the work of teachers in a manner that ensures all courses are based on standards, and that all responsibilities for learning are assigned to one or more “teachers of record” in charge of a student or class. A teacher of record is best understood as person who has a unique identifier (think barcode) for an entire career in teaching. A record is generated whenever a teacher of record has some specified proportion of responsibility for a student’s learning activities. Learning activities must be defined in terms of the performance measures for a particular standard, by subject and grade level.
The TSDL system requires period-by-period tracking of teachers and students every day; including “tests, quizzes, projects, homework, classroom participation, or other forms of day-to-day assessments and progress measures”—a level of surveillance that is said to be comparable to business practices (TSDL, 2011, “Key Components”). http://www.tsdl.org
The system will keep current and longitudinal data on the performance of teachers and individual students, as well schools, districts, states, and educators ranging from principals to higher education faculty. This data will then be used to determine the “best value” investments to make in education and to monitor improvements in outcomes, taking into account as many demographic factors as possible, including health records for preschoolers. A warehouse provides for long-term storage of data to facilitate research on these topics. Of course, USDE also funded statewide longitudinal data systems, complementing the Gates investments.. Now the biggie is one national data systems from infant records through elementary and secondary education, postsecondary education and into the workforce. The unified data system for all that is moving forward. The structure of categories tells what is valued and how to talk about what is valued. The only reference to a civic mission for public education is in the forward from Linda Darling-Hammond. Overall, I think the report is another example of technocratic thinking about accountability. Perhaps that is inevitable if your thinking is constrained by ESSA. Here are links to the education surveillance system now in the works: https://ceds.ed.gov/dataModel.aspx