The final day in the New Mexico’s Teacher Evaluation Lawsuit in Santa Fe was this past Thursday, October 8th. Unfortunately I could not attend day five (or day four) due to a prior travel engagement, but here are two articles that highlight the events of the final day (for now) in court.

The first comes from The Santa Fe New Mexican. This article captures some of the closing arguments on the plaintiffs’ side, primarily in terms of (1) the state model “incorrectly rating nearly 10 percent of the state’s 20,000 teachers,” (2) the state using flawed data, leading to incorrect results, and (3) attempts to convince Judge Thompson to suspend the provision that these teachers — who purportedly post low value-added scores — be put on “growth plans” that lead to termination. As per one of the plaintiffs’ two lawyers: “The mere issue of a notice that you are minimally effective or ineffective puts each district in a position where they can terminate these employees…These are the [high-stakes consequences and] provisions that need to be removed” from New Mexico’s teacher evaluation model.

The main attorney on the defendant’s side avoided talking about #1 and #2 above, and instead focused on #3, “that no teacher had reported losing a job, license or promotion as a result of the teacher evaluations”…yet. In other words, so the defense argued, the system in New Mexico has not (yet) caused “irreparable harm,” although how one defines that is certainly also up for debate (e.g., good teachers leaving teaching because of the system in and of itself might be considered “irreparable harm”).

This defendant’s attorney also focused on what it might mean for the state of New Mexico to lose its No Child Left Behind (NCLB) waiver, which by federal mandate requires New Mexico (and all other states) to have a teacher evaluation system in place as based on student test scores. However, the defendant’s key witness on this topic – Matthew Montaño, Educator Quality Division, New Mexico Public Education Department (PED) – evidently overstated claims about these purported threats while on the stand, after which an expert from Washington DC was phoned in to testify regarding the truth about New Mexico’s waivers; the truth being that approximately a dozen states are thus far not in compliance, and only the state of Washington has thus far lost its waiver. Washington, however, did not lose any federal funding as a result.

The plaintiffs’ attorneys other primary goal was to convince the Judge to file at least a partial injunction now, and then come back to court in April, after both sides have had an equal opportunity to examine the state’s data as per the state’s actual model and model output. The goal here would be to actually determine how the model is functioning and whether it is actually functioning as those on the side of the defense testified (or not, as some of information about the model’s functionality was oddly unknown to the state).

The second article coming from The Albuquerque Journal highlights more details regarding the NCLB waiver, as well as the defendant’s claims regarding “irreparable harm,” or rather that “the plaintiffs cannot show that they were harmed by the teacher evaluation system through loss of a job or reduction of salary…It has to not be theoretical (harm) to these plaintiffs.”

Judge Thomson is expected to present his findings/ruling by early November.

As per my last post about the second day in court in Santa Fe, New Mexico, I was purposefully ambiguous about one of the two articles written in The Santa Fe New Mexican regarding my testimony. The first article titled, “Experts differ on test-based evaluations at NM hearing,” I felt fairly captured the events of the second day in court, but the second article titled, “Professor’s testimony: Teacher eval system ‘not ready for prime time,” did not. But that is about all that I said, being purposefully ambiguous for two reasons that I can now (more or less) share.

The first reason was that the author of this article (in my opinion) unfairly captured my four hours of testimony, by primarily positioning me as an “expert witness” who did not know really anything about the New Mexico teacher evaluation model. Just to be clear, during my testimony I explained that I had not analyzed the actual data from New Mexico. I also argued (but this was unfortunately not highlighted in this particular article), that I could not find anything about the New Mexico model’s output (e.g., indicators of reliability or consistency in terms of teachers’ rankings over time, indicators of validity as per, for example, whether the state’s value-added output correlated, or not, with the other “multiple measures” used in New Mexico’s teacher evaluation system), pretty much anywhere given my efforts.

I testified that this, in and of itself, was problematic, given much of what should have been accessible and retrievable, for example, via the website of New Mexico’s Public Education Department (PED), via the internet in general (e.g., about developer Pete Goldschmidt’s value-added model), and via my attempts through contacts throughout New Mexico to gather the particular information for which I was looking, was not available to the public via any of these means. This, in and of itself, as I also testified, was problematic, especially given New Mexico Governor Martinez’s claims about “Ensuring Transparency and Ethics in our Government.”

Also not explained in this article was that I also analyzed New Mexico’s model using the documents made available via the exhibits submitted for this case, by both the plaintiffs and the defense. What I did not do, however, was conduct any direct analyses of any of the state’s actual data, yet, and for a variety of reasons (e.g., these data are a part of a lawsuit, and I am on the “wrong side,” there are standard data procedural and confidentiality agreements that take considerable time to secure, there is a certain timeline in place). This was also not explained.

I wrote a letter to the editor of The Santa Fe New Mexican in response, but not to my surprise, I never received a response, nor was this letter published. The goal of this letter, though, was to give those in Santa Fe, but also others throughout the state of New Mexico, not only a more comprehensive and accurate assessment of my testimony, but more a note about how the taxpayers of New Mexico have a right to know much, much more about their state’s teacher evaluation model, as well as the model’s output (i.e., to see how the model is actually functioning as claimed).

As I also testified, one of my goals (and I believe this to also be a goal of the plaintiffs’ writ large) is to (a) get the state of New Mexico to release the data to an external evaluator to evaluate the models’ functionality (this person certainly does not have to be me) or (2) release the data to the “expert witnesses” on both the plaintiffs’ side (i.e., me) and the defendants’ side (i.e., Thomas Kane of Harvard), so that we can both examine these data independently, and then come back to the court with our findings and overall assessments regarding the model’s overall strengths and weakness, as per the actual data. 

This brings me to my second reason for being purposefully ambiguous throughout my prior post: The defendants’ “expert witness” Thomas Kane of Harvard. I wanted to wait to be 100% certain that Thomas Kane had not examined any state data before I commented that he would eventually be critiqued for not having done so. But having witnessed his 5.5 hours of testimony yesterday, let me just say it was an interesting 5.5 hours, that (in my opinion) did not work in the defendants’ favor.

Kane, like me, had not examined any of New Mexico’s actual data. This was surprising in the sense that he was actually retained by the state, and his lawyers could have much more likely, and literally, handed him the state’s dataset as their “expert witness,” likely regardless of the aforementioned procedures and security measures (but perhaps not the timeline). Also surprising, though, was that Kane had clearly not examined any of the exhibits submitted for this case, by both the plaintiffs and the defense, either. He was essentially in attendance, on behalf of the state as their “expert witness,” to “only speak to [teacher] evaluations in general.” As per an article in The Albuquerque Journal, he “stressed that numerous studies [emphasis added] show that teachers make a big impact on student success,” expressly contradicting the American Statistical Association (ASA) on the stand, while also (only) referencing (non-representative) studies of primarily his econ-friends (e.g. Raj Chetty, Eric Hanushek, Doug Staiger) and studies of his own (e.g, as per his Measures of Effective Teaching (MET) studies). For more information in general, though, see the full articles in both The Albuquerque Journal and The Santa Fe New Mexican.

As also highlighted in both of these articles is that as one of the defendants’ witnesses, the Superintendent of the Roswell Independent School District testified in favor of the state’s model. As as highlighted in the The Santa Fe New Mexican, he testified that he viewed the new system as “an improvement over past practices [namely the Adequate Yearly Progress (AYP) measures written into No Child Left Behind (NCLB)] because [he believed the new system gave] him more information about his teachers.” He did not, however, testify as to how value-added output gave him this information, as the information about which he spoke was primarily about the observational system now in place, and the conversations surrounding such data, although districts also used similar observational systems prior.

He also testified that, as per the article in The Santa Fe New Mexican, “he had renewed licenses for teachers who received low ratings, despite the state Public Education Department’s protocol…[for one reason being]…he has too few teachers and can’t afford to lose any” due to this system. Related, as per the article in The Albuquerque Journal he continued testifying that “I am down teachers. I don’t need teachers, number 1, quitting over this and, number 2, I am not going to be firing teachers over this.” His district of about 600 teachers currently has approximately 30 open teaching positions, “an unusually high number;” hence, “he would rather work with his current staff than bring on substitutes” in compliance. So while he testified on behalf of the state, he also testified he was not necessarily in favor of the consequences being attached to the state’s teacher evaluation output, even if as currently being positioned by the defense as “low-stakes.”

It was more than an interesting day in court indeed. Stay tuned for Day Four (although due to prior conflicts, I will not be in attendance). I will still report on it, though, as best I can.