Following up on two of my most recent posts, the first about Commissioner Huffman’s (un)inspiring TEDxNashville talk in which he vociferously celebrated Tennessee students’ recent (albeit highly questionable) gains on the National Assessment of Education Progress (NAEP) scores, and the second about Huffman’s (and the Tennessee Department of Education’s) unexpected postponement of the release of its state-level (TCAP) standardized test scores — test scores that were, by law, to account for 15 to 25 percent of Tennessee students’ final grades — it seems a few more “behind the scenes” details surrounding what is going on in Tennessee might also explain the state’s current situation (see other “behind the scenes” details in the first aforementioned post).
We also now know that in Tennessee, on the grade 12 NAEP the state ranks among the lowest in the nation. We also now know that in Tennessee, before the 4th grade NAEP tests were taken, the state withheld an inordinate proportion of (low-scoring) students in the 3rd grade which (likely) caused (or at the very least helped to produce) the (purportedly) artificial gains observed in grade 4. Tennessee is not the first to have done this, however (see Boston College Professor Walter Haney’s article about the “Texas Miracle” that also occurred on then Governor George W. Bush’s watch here).
These “behind the scenes” explanations, unfortunately for Huffman, explain more of the gains than the casual observer might realize, although Huffman is likely counting on only casual observations being made, as it is this “unassailable evidence [emphasis added]…that [should] carry the day.” Right?!?
Now, a (perhaps reasonable) conspiracy theory has also emerged. As per this blog post, a multitude of (perhaps reasonable) reasons are offered that should “raise even more questions about the motives and reasons behind the non-delivery of [the state’s] TCAP scores to schools across the state.” Possible alternative explanations, besides the data just not being ready, given the state was trying to “narrow” and better align its tests to the forthcoming Common Core (and its “post-equating” methods took longer than anticipated), include the following…as written by the author of this same post:
- “Narrowing” sounds like “erasing” certain questions and “post-equating” sounds like their version of “post-dating” a check. Taken together they sound very much like cooking the numbers. By eliminating certain questions after the fact, Huffman & Co. will be able to take out those questions where students consistently got wrong answers, thereby lifting the student’s overall score. The state asserts they made the decision to do this before this week, but they conveniently leave out WHO made the decision and WHEN the decision was made. They also do not tell the public WHO will be making the decisions to “narrow” (i.e., take out) certain questions. Who decides? Pearson consultants? Huffman lackey’s? Who?”
As noted in the second of the aforementioned VAMboozled! posts, “the state made these changes without keeping districts informed about the changes that were being made.” In addition, they did this without the teams of folks (e.g., practitioners) who are or who really should always be involved in such processes, in particular to make sure that at the very least the content of the test items is still (and I use this term loosely) valid. So why would these folks not be involved? Perhaps because it is TRUE that Huffman et al. could indeed significantly skew his state’s TCAP test scores upwards by tossing out questions that did not represent Tennessee’s purported and widely celebrated “progress” in as positive of a light. While in a related op-ed piece, “the state” mentions they have a TAC [Technical Advisory Committee] on board, which is common, there is no mention regarding who comprises this group or whether they were involved with the delayed release or “the state’s” reasoning behind it. This, too, is essential information (suspiciously) not provided.
Regardless, this type of Machiavellian method works quite well, and has proven itself worthy for more than 30 years (of which I am aware) when state-level “leaders” desire to artificially inflate their (most often) state-level test scores.
More specifically, this most often occurs when tests are made easier “behind the scenes” so that more students pass state-level tests over time. This is most often done because state’s (and state leaders) cannot reasonably fail “too many” students. This happens…when tests are administered and “too many” students often fail…which leads to newspaper headlines about students in states not academically proficient (as uber-arbitrarily defined)…which leads to easier tests after the most difficult items are removed from the tests…which yields new headlines that the students and their teachers are taking more seriously the standards, hence the observed increases…which leads to more “behind the scenes” manipulation…which again leads to higher observed test scores…which ultimately leads to a new state leader/politician calling for higher standards because “too many” students are passing the tests…which leads to higher standards and new and improved tests to hold educators accountable for meeting higher standards…which leads to a continuous repeat cycle.
This is what we in the testing/assessment field call “the saw-tooth effect or phenomenon,” (see more about this, as per University of Colorado – Boulder Professor Emeritus Robert Linn here). This (typically) only occurs with state-level tests that are highly manipulable, both “behind the scenes” but also in practice (e.g., via “teaching to the test”).
…back to the prior post:
- “So now that we have established a Method, how about a Motive? Here is where it gets really interesting and raises possibilities of scandal, cover-your-ass politics and even possible criminal activity. In politics timing is everything. So the question must be asked, WHAT was the reason the state deliberately refused to inform schools about their “new” methods? WHAT was occurring at precisely the same time the state [Department of Education] was making their last-second announcement about not delivering TCAP scores? If the unaltered, un-narrowed, un-post-equated TCAP scores were really bad, and were released as required by law…what would have been the result?” Why such the concern? Here is one possible answer: Arne Duncan was coming to town. “The [Tennessee] governor and Huffman were hosting 400 education writers and [Duncan]…the Obama Secretary of Education in Nashville…the exact time TCAP scores were supposed to be delivered to the schools. Can you imagine how embarrassing it would have been if 400 education writers from around the country were exposed to horrible test scores from Tennessee’s much-claimed Common Core “success”? The governor and Huffman would look like fools AND liars. And the public may have caught onto their scam. And THAT, ladies & gentlemen of the jury, is what you would need to convict someone of mis-feasance, mal-feasance, stupid-feasance and bend-over-here-it-comes-feasance.”
While I am not a conspiracy theorist, and I like to believe people and politicians are at least born of good nature, I also know that we have seen cases like this all too often in the past, WHEN test scores matter and ESPECIALLY WHEN politicians have their political lives and futures on the “results trump all” line. It is too often the case that they “must” engage in Machiavellian behaviors to save, if not extend their political careers.
On that note, in the aforementioned op-ed piece another author has come to a similar conclusion for what it’s worth, writing: “I think this year’s TCAP had multiple mistakes on it, and as state officials gasp in growing horror at low TCAP scores, they’re backpedaling, waiver-giving and post-equating their way [out of] a bureaucratic nightmare.” The author of this piece went as far as asking for Huffman’s resignation.
That being said, I will leave you all to be the judges here, as I have offered everything I have given this (crazy, but in many ways unfortunately familiar) situation in Tennessee. Those of you in Tennessee — please add to the discussion as well.
