Evidence of Grade and Subject-Level Bias in Value-Added Measures: Article Published in TCR

ShareTweet about this on TwitterShare on Facebook26Email this to someoneShare on Google+1Share on LinkedIn2Share on Reddit0

One of my most recent posts was about William Sanders — developer of the Tennessee Value-Added Assessment System (TVAAS), which is now more popularly known as the Education Value-Added Assessment System (EVAAS®) — and his forthcoming 2015 James Bryant Conant Award — one of the nation’s most prestigious education honors, that will be awarded to him this next month by the Education Commission of the States (ECS).

Sanders is to be honored for his “national leader[ship] in value-added assessments, [as] his [TVAAS/EVAAS] work has [informed] key policy discussion[s] in states across the nation.”

Ironically, this was announced the same week that one of my former doctoral students — Jessica Holloway-Libell, who is soon to be an Assistant Professor at Kansas State University — had a paper published in the esteemed Teachers College Record about this very model. Her paper titled, “Evidence of Grade and Subject-Level Bias in Value-Added Measures” can be accessed (at least for the time being) here.

You might also recall this topic, though, as we posted her two initial drafts of this article over one year ago, here and here. Both posts followed the analyses she conducted after a VAMboozled follower emailed us expressing his suspicions about grade and subject area bias in his district in Tennessee, in which he was/still is a school administrator.  The question he posed was whether his suspicions were correct, and whether this was happening elsewhere in his state, using Sanders’ TVAAS/EVAAS model.

Jessica found it was.

More specifically, Jessica found that:

  1. Teachers of students in 4th and 8th grades were much more likely to receive positive value-added scores than in other grades (e.g., 5th, 6th, and 7th grades); hence, that 4th and 8th teachers are generally better teachers in Tennessee using the TVAAS/EVAAS model.
  2. Mathematics teachers (theoretically throughout Tennessee) are, overall, more effective than Tennessee’s English/language arts teachers, regardless of school district; hence, mathematics teachers are generally better than English/language arts teachers in Tennessee using the TVAAS/EVAAS model.

Being a former mathematics teacher myself, I’d like to support the second claim as being true, being subject-area biased myself. But the fact of the matter is that the counterclaims in this case are obviously true, likely entirely, instead.

It’s not that either or any set of these teachers are in fact better, it’s that Sanders’ TVAAS/EVAAS model – — the model for which Sanders is receiving this esteemed award — is yielding biased output. It is doing this for whatever reason (e.g., measurement error, test construction) but this just adds to the list of other problems (see, for example, here, here, and here) and quite frankly the reasons why this model, not to mention its master creator, is undeserving of really any award, except for a Bunkum, perhaps.

ShareTweet about this on TwitterShare on Facebook26Email this to someoneShare on Google+1Share on LinkedIn2Share on Reddit0

One thought on “Evidence of Grade and Subject-Level Bias in Value-Added Measures: Article Published in TCR

  1. Thank you for sharing this – I would not be surprised if there is deep and widespread bias in all VA models. In my district in Phoenix, we noticed our Value-Added scores were skewed for our Special Areas (electives) and Special Ed teachers … significantly higher numbers of “ineffective” or “developing” teachers in those content areas, versus general ed K-8 teachers. Unfortunately, there is no transparency from the agency calculating our scores to discover any further biases, such as by grade level, school site, etc. Teachers are often powerless to find and use this kind of data (although many of us instinctively know that these VA models are not valid, and especially not uniformly valid for all grade levels/content areas!), so it is essential that research like yours and like Jessica’s is happening. You give us hope!

Leave a Reply

Your email address will not be published. Required fields are marked *