Please follow and like us:

As per my last post about the second day in court in Santa Fe, New Mexico, I was purposefully ambiguous about one of the two articles written in The Santa Fe New Mexican regarding my testimony. The first article titled, “Experts differ on test-based evaluations at NM hearing,” I felt fairly captured the events of the second day in court, but the second article titled, “Professor’s testimony: Teacher eval system ‘not ready for prime time,” did not. But that is about all that I said, being purposefully ambiguous for two reasons that I can now (more or less) share.

The first reason was that the author of this article (in my opinion) unfairly captured my four hours of testimony, by primarily positioning me as an “expert witness” who did not know really anything about the New Mexico teacher evaluation model. Just to be clear, during my testimony I explained that I had not analyzed the actual data from New Mexico. I also argued (but this was unfortunately not highlighted in this particular article), that I could not find anything about the New Mexico model’s output (e.g., indicators of reliability or consistency in terms of teachers’ rankings over time, indicators of validity as per, for example, whether the state’s value-added output correlated, or not, with the other “multiple measures” used in New Mexico’s teacher evaluation system), pretty much anywhere given my efforts.

I testified that this, in and of itself, was problematic, given much of what should have been accessible and retrievable, for example, via the website of New Mexico’s Public Education Department (PED), via the internet in general (e.g., about developer Pete Goldschmidt’s value-added model), and via my attempts through contacts throughout New Mexico to gather the particular information for which I was looking, was not available to the public via any of these means. This, in and of itself, as I also testified, was problematic, especially given New Mexico Governor Martinez’s claims about “Ensuring Transparency and Ethics in our Government.”

Also not explained in this article was that I also analyzed New Mexico’s model using the documents made available via the exhibits submitted for this case, by both the plaintiffs and the defense. What I did not do, however, was conduct any direct analyses of any of the state’s actual data, yet, and for a variety of reasons (e.g., these data are a part of a lawsuit, and I am on the “wrong side,” there are standard data procedural and confidentiality agreements that take considerable time to secure, there is a certain timeline in place). This was also not explained.

I wrote a letter to the editor of The Santa Fe New Mexican in response, but not to my surprise, I never received a response, nor was this letter published. The goal of this letter, though, was to give those in Santa Fe, but also others throughout the state of New Mexico, not only a more comprehensive and accurate assessment of my testimony, but more a note about how the taxpayers of New Mexico have a right to know much, much more about their state’s teacher evaluation model, as well as the model’s output (i.e., to see how the model is actually functioning as claimed).

As I also testified, one of my goals (and I believe this to also be a goal of the plaintiffs’ writ large) is to (a) get the state of New Mexico to release the data to an external evaluator to evaluate the models’ functionality (this person certainly does not have to be me) or (2) release the data to the “expert witnesses” on both the plaintiffs’ side (i.e., me) and the defendants’ side (i.e., Thomas Kane of Harvard), so that we can both examine these data independently, and then come back to the court with our findings and overall assessments regarding the model’s overall strengths and weakness, as per the actual data. 

This brings me to my second reason for being purposefully ambiguous throughout my prior post: The defendants’ “expert witness” Thomas Kane of Harvard. I wanted to wait to be 100% certain that Thomas Kane had not examined any state data before I commented that he would eventually be critiqued for not having done so. But having witnessed his 5.5 hours of testimony yesterday, let me just say it was an interesting 5.5 hours, that (in my opinion) did not work in the defendants’ favor.

Kane, like me, had not examined any of New Mexico’s actual data. This was surprising in the sense that he was actually retained by the state, and his lawyers could have much more likely, and literally, handed him the state’s dataset as their “expert witness,” likely regardless of the aforementioned procedures and security measures (but perhaps not the timeline). Also surprising, though, was that Kane had clearly not examined any of the exhibits submitted for this case, by both the plaintiffs and the defense, either. He was essentially in attendance, on behalf of the state as their “expert witness,” to “only speak to [teacher] evaluations in general.” As per an article in The Albuquerque Journal, he “stressed that numerous studies [emphasis added] show that teachers make a big impact on student success,” expressly contradicting the American Statistical Association (ASA) on the stand, while also (only) referencing (non-representative) studies of primarily his econ-friends (e.g. Raj Chetty, Eric Hanushek, Doug Staiger) and studies of his own (e.g, as per his Measures of Effective Teaching (MET) studies). For more information in general, though, see the full articles in both The Albuquerque Journal and The Santa Fe New Mexican.

As also highlighted in both of these articles is that as one of the defendants’ witnesses, the Superintendent of the Roswell Independent School District testified in favor of the state’s model. As as highlighted in the The Santa Fe New Mexican, he testified that he viewed the new system as “an improvement over past practices [namely the Adequate Yearly Progress (AYP) measures written into No Child Left Behind (NCLB)] because [he believed the new system gave] him more information about his teachers.” He did not, however, testify as to how value-added output gave him this information, as the information about which he spoke was primarily about the observational system now in place, and the conversations surrounding such data, although districts also used similar observational systems prior.

He also testified that, as per the article in The Santa Fe New Mexican, “he had renewed licenses for teachers who received low ratings, despite the state Public Education Department’s protocol…[for one reason being]…he has too few teachers and can’t afford to lose any” due to this system. Related, as per the article in The Albuquerque Journal he continued testifying that “I am down teachers. I don’t need teachers, number 1, quitting over this and, number 2, I am not going to be firing teachers over this.” His district of about 600 teachers currently has approximately 30 open teaching positions, “an unusually high number;” hence, “he would rather work with his current staff than bring on substitutes” in compliance. So while he testified on behalf of the state, he also testified he was not necessarily in favor of the consequences being attached to the state’s teacher evaluation output, even if as currently being positioned by the defense as “low-stakes.”

It was more than an interesting day in court indeed. Stay tuned for Day Four (although due to prior conflicts, I will not be in attendance). I will still report on it, though, as best I can.