Just this week, P. L. Thomas, an Associate Professor at Furman University, wrote a very good piece about VAMs, some of their philosophical and historical underpinnings, and some of the recent studies/papers released about VAMs including the recent American Statistical Association (ASA) Position Statement (reviewed in this blog here and just yesterday here), the recent work of Edward Heartel (covered in this blog here, here, and here), some of the wisdom of Rutgers Professor Bruce Baker (mentioned in this blog here and here), and the outstanding, I mean outlandish, Chetty et al. study (covered in this blog here and here). Thanks to my former colleague for sending this along!
Anyhow, here is the direct link to the actual post, although I have also re-posted the initial piece below, especially for those of you reading via your handheld’s and/or emails. Do visit the actual post, though, and perhaps follow his blog, as a quick perusal of the blog’s content will likely evidence that there is some other good content herein.
But for this post, specifically, Thomas writes:
“Devaluing Teachers in the Age of Value-Added”
“We teach the children of the middle class, the wealthy and the poor,” explains Anthony Cody, continuing:
We teach the damaged and disabled, the whole and the gifted. We teach the immigrants and the dispossessed natives, the transients and even the incarcerated.
In years past we formed unions and professional organizations to get fair pay, so women would get the same pay as men. We got due process so we could not be fired at an administrator’s whim. We got pensions so we could retire after many years of service.
But career teachers are not convenient or necessary any more. We cost too much. We expect our hard-won expertise to be recognized with respect and autonomy. We talk back at staff meetings, and object when we are told we must follow mindless scripts, and prepare for tests that have little value to our students.
During the 1980s and 1990s, U.S. public schools and the students they serve felt the weight of standards- and test-based accountability—a bureaucratic process that has wasted huge amounts of tax-payers’ money and incalculable time and energy assigning labels, rankings, and blame. The Reagan-era launching of accountability has lulled the U.S. into a sort of complacency that rests on maintaining a gaze on schools, students, and test data so that no one must look at the true source of educational failure: poverty and social inequity, including the lingering corrosive influences of racism, classism, and sexism.
The George W. Bush and Barack Obama eras—resting on intensified commitments to accountability such as No Child Left Behind (NCLB) and Race to the Top (RTTT)—have continued that misguided gaze and battering, but during the past decade-plus, teachers have been added to the agenda.
As Cody notes above, however, simultaneously political leaders, the media, and the public claim that teachers are the most valuable part of any student’s learning (a factually untrue claim), but that high-poverty and minority students can be taught by those without any degree or experience in education (Teach for America) and that career teachers no longer deserve their profession—no tenure, no professional wages, no autonomy, no voice in what or how they teach.
And while the media and political leaders maintain these contradictory narratives and support these contradictory policies, value-added methods (VAM) of evaluating and compensating U.S. public teachers are being adopted, again simultaneously, as the research base repeatedly reveals that VAM is yet another flawed use of high-stake accountability and testing.
When Raj Chetty, John N. Friedman, and Jonah E. Rockoff released (and re-released) reports claiming that teacher quality equates to significant earning power for students, the media and political leaders tripped over themselves to cite (and cite) those reports.
What do we know about the Chetty, et al., assertions?
[T]hose using the results of this paper to argue forcefully for specific policies are drawing unsupported conclusions from otherwise very important empirical findings. (Di Carlo)
These are interesting findings. It’s a really cool academic study. It’s a freakin’ amazing data set! But these findings cannot be immediately translated into what the headlines have suggested – that immediate use of value-added metrics to reshape the teacher workforce can lift the economy, and increase wages across the board! The headlines and media spin have been dreadfully overstated and deceptive. Other headlines and editorial commentary has been simply ignorant and irresponsible. (No Mr. Moran, this one study did not, does not, cannot negate the vast array of concerns that have been raised about using value-added estimates as blunt, heavily weighted instruments in personnel policy in school systems.) (Baker)
And now, a thorough review concludes:
Can the quality of teachers be measured the way that a person’s weight or height is measured? Some economists have tried, but the “value-added” they have attempted to measure has proven elusive. The results have not been consistent over tests or over time. Nevertheless, a two-part report by Raj Chetty and his colleagues claims that higher value-added scores for teachers lead to greater economic success for their students later in life. This review of the methods of Chetty et al. focuses on their most important result: that teacher value-added affects income in adulthood. Five key problems with the research emerge. First, their own results show that the calculation of teacher value-added is unreliable. Second, their own research also generated a result that contradicts their main claim—but the report pushed that inconvenient result aside. Third, the trumpeted result is based on an erroneous calculation. Fourth, the report incorrectly assumes that the (miscalculated) result holds across students’ lifetimes despite the authors’ own research indicating otherwise. Fifth, the report cites studies as support for the authors’ methodology, even though they don’t provide that support. Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy.
Similar to the findings in Edward H. Haertel’s analysis of VAM, Reliability and validity of inferences about teachers based on student test scores (ETS, 2013), the American Statistical Association has issued ASA Statement on Using Value-Added Models for Educational Assessment, emphasizing:
Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.
The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data.
Among DiCarlo, Baker, Haertel and the ASA, several key patterns emerge regarding VAM: (1) VAM remains an experimental statistical model, (2) VAM is unstable and significantly impacted by factors beyond a teacher’s control and beyond the scope of that statistical model to control, and (3) implementing VAM in high-stakes policies exaggerates the flaws of VAM.
The rhetoric about valuing teachers rings hollow more and more as teaching continues to be dismantled and teachers continue to be devalued by misguided commitments to VAM and other efforts to reduce teaching to a service industry.
VAM as reform policy, like NCLB, is sham-science being used to serve a corporate need for cheap and interchangeable labor. VAM, ironically, proves that evidence does not matter in education policy.
Like all workers in the U.S., we simply do not value teachers.
Political leaders, the media, and the public call for more tests for schools, teachers, and students, but they continue to fail themselves to acknowledge the mounting evidence against test-based accountability.
And thus, we don’t need numbers to prove what Cody states directly: “But career teachers are not convenient or necessary any more.”