In Response to a Tennessee Assistant Principal’s Concerns – Part II Mathematics

ShareTweet about this on TwitterShare on Facebook0Email this to someoneShare on Google+0Share on LinkedIn0Share on Reddit0

In our most recent post, we conducted the first of two (this is the second) follow-up analyses to examine the claims put forth in another recent post about a Tennessee assistant principal’s suspicions regarding his state of Tennessee’s value-added scores, as measured by the Tennessee Value-Added Assessment System (TVAAS; publicly available here). This is the final installment of this series, but this time we focus on the mathematics value-added scores and, probably not surprising, illustrate below quite similar results.

Again, we analyzed data from the 10 largest school districts in the state based on population data (found here). We looked at grade-level value-added scores for 3rd through 8th grade mathematics. For each grade level, we calculated the percentage of schools per district that had positive value-added scores for 2013 (see Table 1) and for their three-year composite scores, since these are often considered more reliable (see Table 2).

We were particularly interested in knowing how likely a grade level was to get a positive value-added score and to see if there were any trends across districts. Consistent with our English/Language Arts (ELA) findings, similar trends in mathematics were apparent as well.  For clarity purposes, and as we did with the ELA findings, we color-coded the chart—green signifies the grade levels for which 75% or more of the schools received a positive value-added score, while red signifies the grade levels for which 25% or less of the schools received a positive value-added score.

Table 1: Percent of Schools that had Positive Value-Added Scores by Grade and District (2013) Mathematics

District 3rd  Grade 4th   Grade 5th   Grade 6th  Grade
7th Grade 8th Grade
Memphis 42% 91% 70% 37% 84% 55%
Nashville-Davidson NA 60% 65% 70% 90% 44%
Knox 72% 87% 57% 64% 93% 67%
Hamilton 31% 86% 84% 38% 76% 78%
Shelby 97% 97% 61% 75% 100% 69%
Sumner 81% 77% 50% 67% 75% 58%
Montgomery NA 86% 71% 14% 86% 57%
Rutherford 79% 100% 67% 62% 77% 85%
Williamson NA 67% 21% 100% 100% 56%
Murfreesboro NA 100% 60% 90% NA* NA*

Though the mathematics scores were not as glaringly biased as the ELA scores, there were some alarming trends to notice. In particular, the 4th and 7th grade value-added scores were consistently higher than those of the 3rd, 5th, 6th, and 8th grades in mathematics, which had much greater variation across districts. In fact, all districts had at least 75% of their schools receive positive value-added scores in 7th grade and at least 60% in fourth grade. To recall, the seventh grade scores in ELA were drastically lower, with all districts having no more than 50% of their schools receive positive value-added scores (five of which had fewer than 25%). The 6th and 8th grades also had more variation in mathematics than in ELA.

Table 2: Percent of Schools that had Positive Value-Added Scores by Grade and District (Three-Year Composite) Mathematics

District 3rd Grade 4th  Grade 5th  Grade
6th  Grade 7th  Grade
8th  Grade
Memphis NA 90% 72% 62% 96% 78%
Nashville-Davidson NA 73% 67% 89% 97% 74%
Knox NA 96% 79% 90% 93% 93%
Hamilton NA 89% 80% 52% 90% 71%
Shelby NA 97% 79% 75% 100% 69%
Sumner NA 92% 65% 90% 92% 100%
Montgomery NA 95% 85% 71% 86% 71%
Rutherford NA 100% 79% 69% 92% 85%
Williamson NA 91% 22% 100% 100% 89%
Murfreesboro NA 100% 90% 100% NA NA

As for the three-year composites scores, schools across the state were much more likely to receive positive value-added scores than negative value-added scores in all tested grade levels. This, compared to the ELA scores where 6th and 7th grades struggled to earn positive scores, suggests that there is some level of subject bias going on here. Specifically, a majority of schools across all districts received positive value-added scores at each grade level for mathematics, with the small exception of 5th grade in one school district. For 4th and 7th grades, almost every school received positive scores.

Again, of most importance here is how we choose to interpret these results. By Tennessee’s standard (given their heavy reliance on the TVAAS to evaluate teachers), our conclusion would be that the mathematics teachers are, overall, more effective than the ELA teachers in almost every tested grade level (with the exception of 8th grade ELA), regardless of school district.

Perhaps a more reasonable explanation, though, is that there is some bias in the tests upon which the TVAAS scores are measured (as likely related to some likely issues with the vertical scaling of Tennessee’s tests, not to mention other measurement errors). Far more students across the state  demonstrated growth in mathematics than in ELA (for the past three years at least). To simply assume that this is caused by teacher effectiveness is crass at best.

 

Analysis conducted by Jessica Holloway-Libell

ShareTweet about this on TwitterShare on Facebook0Email this to someoneShare on Google+0Share on LinkedIn0Share on Reddit0

One thought on “In Response to a Tennessee Assistant Principal’s Concerns – Part II Mathematics

  1. These findings are compelling. If these data can be dug deeper, beyond the percentage of positive or negative scores, they may reveal more valuable information. The value-added scores of Grade 3 in Arizona seem different from that of other grades, though I haven’t done any analysis.

Leave a Reply

Your email address will not be published. Required fields are marked *