The model I know best, as I have been researching this one for now almost a decade, is the TVAAS (which is now more popularly known as the EVAAS) which, as mentioned numerous times on this blog, has its strong roots in Tennessee. It is in Tennessee that William L. Sanders, a then (in the 1980s/90s) Adjunct Professor of Agriculture at the University of Tennessee – Knoxville, developed the TVAAS.
Contrary to what was written in an article released today in The Tennessean, however, he did not invent “value-added.” This has been a mainly econometric approach that can be found in economics literature since the 1970s. Regardless, it is worth a read of this article to understand this model’s history, the model in education from which much of the current education system’s value-added “craze” (or as they call it “vogue” trend) came, as this article was written in response to the many lawsuits coming to fruition in Tennessee (see posts forthcoming this week), largely in the model’s defense.
Interesting points to point out:
- The state of Tennessee pays $1.7 million annually to SAS to calculate their TVAAS scores, and to also put them online. This is certainly a nice chunk of taxpayer funds per year, also considering that the states of Ohio, Pennsylvania and North Carolina use the EVAAS statewide as well.
- “Sanders dismisses claims of volatility,” when the research evidence coming from the EVAAS system continues to warrant the counter claims; that is, that this model is about as volatile as the rest of the models that are currently available and operational.
- Sanders claims that the model’s “formula adjusts for one-offs,” again contrary to what continues to emerge in the research. The statistics, unfortunately, do not work like a pure statistician would (still) hope.
- Sanders claims the model also “overcomes measures that track achievement alone, such as students’ race or family income” which makes issues of “bias” — a topic of recent posts here — null and void. This, too, is becoming increasingly evident in the research literature, as also evidenced by some of our strongest statisticians working in these areas (e.g., Mark Reckase at Michigan State University).
- Sanders and others (including state leaders and even U.S. Secretary of Education, Arne Duncan) have lauded Tennessee’s use of accountability instruments, like the TVAAS, for Tennessee’s large gains on the most recent National Assessment of Educational Progress (NAEP) outcomes. Others, however, have cautioned against this celebration because 1) the results mask the expanding achievement gap in Tennessee; 2) the state’s lowest socioeconomic students continue to perform poorly on the test; 3) Tennessee didn’t make gains significantly different than many other states; and 4) other states with similar accountability instruments and policies (e.g., Colorado, Louisiana) did not make similar gains, while states without such instruments and policies (e.g., Kentucky, Iowa, Washington) did make similar gains (I should add that Kentucky’s achievement gap is also narrowing and their lowest socioeconomic students have made significant gains). Claiming that NAEP scores increased because of TVAAS-use and other stringent accountability policies is completely unwarranted (see also this article in Education Week).
No surprise, I guess, that none of the other “issues” about which the EVAAS has been continuously questioned and critiqued were addressed in this article (e.g., about fairness and the teachers who are not TVAAS eligible, validity or the lack of relationships between the TVAAS and other indicators of quality in Tennessee, subject and grade level bias, as written about here and here, etc.).
What does adjust for one-off mean. I need a min-lesson.
Here is more detail from his actual quote in the article: “…the formula adjusts for one-offs, such as a smart kid who bombs on a testing day or a teacher who cheats to get a bonus.” He’s referring to them as “one-offs” rather than, perhaps, outliers or extremes or students who test under random/systematic circumstances that often cause error. This model can control for that…too 😉
I too have research VAM for a number of years now. One critical issue that must not be lost in the debate is that VAM is based upon standardized achievement test scores, which are demonstrably not psychometrically valid. The most critical assumption for VA to work is that the input be clearly valid and reliable. There are also a number of other assumptions the metrics of VAM fail to reconcile. It is a horribly bad system to use to evaluate educators.