Bennett Mackinney, a local administrator and former doctoral student in one of my research methods courses, recently emailed me asking the following: “Is a valid and reliable VAM theoretically possible? If so, is the issue that no state is taking the time and energy to develop it correctly? Or, is it just impossible? I need to land on a pragmatic position on VAMs. Part of the problem is that for the past decade all of us over here at Title I schools [i.e., schools with disproportionate numbers of disadvantaged students] have been saying “[our state test] is not fair…. our kids come to us with so many issues… you need to measure us on growth not final performance….” I feel like VAM advocates will come to us and say, “fine, here’s a model that will meet your request to be fairly measured on growth…”
I responded with the following: “In my research-based opinion, we are searching and always will be searching for a type of utopia in this area, one that will likely be out of our research forever UNLESS these PARC, etc. tests come through with miracles [which, as history is likely to repeat itself, is highly unlikely]. However, at the end of the day, [we] can be confident that this is better than the snapshot measures used before (I.e., before growth measures), particularly for analyses of large-scale trends, but certainly not for teacher evaluation and especially not for high-stakes teacher evaluation purposes. Regardless of the purpose, though, NEVER should [VAMs] be used in isolation of other measures and NEVER should they be used without a great deal of human judgment re: what the VAM estimates in reality and in context demonstrate, in light of what they can and just cannot do.”
For more on this, see my prior post about the position statement recently released by the American Statistical Association (ASA), or the actual statement itself.
