Here is another post from our “Concerned New Mexico Parent” (see prior posts from him/her here and here). This one is about New Mexico’s A-F School Grading System and how it is not only contradictory, within and beyond itself, but how it also provides little instrumental value to the public as an invalid indicator of the “quality” of any school.
(S)he writes:
- What do you call a high school that has only 38% of its students proficient in reading and 35% of its students proficient in mathematics?
- A school that needs help improving their student scores.
- What does the New Mexico Public Education Department (NMPED) call this same high school?
- A top-rated “A” school, of course.
Readers of this blog are familiar with the VAMs being used to grade teachers. Many states have implemented analogous formulas to grade entire schools. This “forgotten” VAM suffers from all of the familiar problems of the teacher formulas — incomprehensibility, lack of transparency, arbitrariness, and the like.
The first problem with the A-F Grading System is inherent in its very name. The “A-F” terminology implies that this one static assessment is an accurate representation of a school’s quality. As you will see, it is nothing of the sort.
The second problem with the A-F Grading System is that is is composed of benchmarks that are arbitrarily weighted and scored by the NMPED using VAM methodologies.
Thirdly, the “collapsing of the data” from a numeric score to a grade (corresponding to a range of values) causes valuable information to be lost.
Table 1 shows the range of values for reading and mathematics proficiencies for each of the five A-F grade categories for New Mexico schools.
Table 1: Ranges and Median of Reading and Mathematics Proficiencies by A-F School Grade
| School A-F Grade | Number of Schools |
Reading Median |
Mathematics Proficiency Range Median |
| A | 86 |
37.90 – 94.00 66.16 |
31.50 – 95.70 58.95 |
| B | 237 |
16.90 – 90.90 58.00 |
4.90 – 90.90 51.30 |
| C | 177 |
0.00 – 83.80 46.30 |
0.00 – 76.20 38.00 |
| D | 21 |
4.50 – 64.60 40.70 |
2.20 – 70.00 31.80 |
| F | 88 |
7.80 – 52.30 31.85 |
3.30 – 40.90 23.30 |
For example, to earn an A rating, a school can have between 37.9% and 94.0% of its students proficient in reading. In other words, a school can have roughly two-thirds of its students fail reading proficiency yet be rated as an “A” school!
The median value listed shows the point which splits the group in half — one-half of the scores are below the median value. Thus, an “A” school median of 66.2% indicates that one-half of the “A” schools have a reading proficiency below 66.2%. In other words, in one-half of the “A” schools 1/3 or more of their students are NOT proficient in reading!
Amazingly, the figures for mathematics are even worse, the minimum proficiency for a B rating is only 4.9% proficient! Scandalous!
Obviously, and contrary to popular and press perceptions, the A-F Grading System has nothing to do with the actual or current quality of the school!
A few case studies will highlight further absurdities of the New Mexico A-F School Grading System next.
Case Study 1 – Highest “A” vs. Lowest “A” High School
Logan High School, Logan, New Mexico received the lowest reading proficiency of any “A” school, and the Albuquerque Institute of Math and Science received the highest reading proficiency score.
These two schools have both received an “A” rating. The Albuquerque Institute had a reading proficiency of 94% and a mathematics proficiency rating of 93%. Logan HS had a reading proficiency of only 38% and a mathematics proficiency rating of only 35%!
How is that possible?
First, much of the A-F VAM, like the teacher VAM is based on multi-year growth calculations and predictions. Logan has plenty of opportunity for growth whereas the Math Academy has “maxed” out most of its scores. Thus, the Albuquerque Institute is penalized in a manner analogous to Gifted and Talented teachers when teacher-level VAM is used. With already excellent scores, there is little, if any, room for improvement.
Second, Logan has an emphasis on shop/trade classes which yields a very high college and CAREER readiness score for the VAM calculation.
Also, a final factor is that the NMPED-defined range for an “A” extends from 75 to 100 points, and Logan barely made it into the A grouping.
Thus, a proficiency score of only 37.9% is no deterrent to an A score for Logan High.
Case Study 2: Hanging on by a Thread
As noted above, any school that scores between 75 and 100 points is considered an “A” school.
This statistical oddity was very beneficial to Hagerman High (Hagerman, NM) in their 2014 School Grade Report Card. They fell 5.99 points overall from the previous year’s score, but they managed to still receive an “A” score since their resulting 2014 score was exactly 75.01.
With this one one-hundredth of a point, they are in the same “A” grade category as the Albuquerque Institute of Math and Science (rated best in New Mexico by NMPED) and the Cottonwood Classical Preparatory School of Albuquerque (rated best in New Mexico by US News).
Case Study 3: A Tale of Two Ranking Systems
This inaccuracy and arbitrariness of any A-F School Grading System was also apparent in a recent Albuquerque Journal News article (May 14, 2015) which reported on the most recent US News ratings of high schools nationwide.
The Journal reported on the top 12 high schools in New Mexico as rated by US News. It is not surprising that most were NMPED A-rated schools. What was unusual is that the 3rd and 5th US News highest rated schools in New Mexico (South Valley Academy and Albuquerque High, both in Albuquerque) were actually rated as B schools by the NMPED A-F School Grading System.
According to NMPED data, I tabulated at least forty-four (44) high schools that were rated as “A” schools with higher NMPED scores than South Valley Academy which had an NMPED score of 71.4.
None of these 44 higher NMPED scoring schools were rated above South Valley Academy by US News.
Case Study 4: Punitive Grading
Many school districts and school boards throughout New Mexico have adopted policies that prohibit punitive grading based on behavior. It is no longer possible to lower a student’s grade just because of their behavior. The grade should reflect classroom assessment only.
NMPED ignores this policy in the context of the A-F School Grading System. Schools were graded down one letter grade if they did not achieve 95% participation rates.
One such school was Mills Elementary in the Hobbs Municipal Schools District. Only 198 students were tested; they fell 11 short of the 95% mark and were penalized one “grade”-level. Their grade was reduced from a “D” to an “F” In fact, Mills Elementary proficiency scores were higher than the A-rated Logan High School discussed earlier.
The likely explanation is that Hobbs has a highly transient population with both seasonal farm laborers and oil-field workers predominating in the local economy.
For more urban schools, it will be interesting to see how the NMPED policy of punitive grading will play out with the increasingly popular Opt-Out movement.
Conclusion
It is apparent that the NMPED’s A-F School Grading System rates schools deceptively using VAM-augmented data and provides little of any value to the public as to the “quality” of a school. By presenting it in the form of an “NMPED School Grade Report Card” the state seeks to hide its arbitrary nature.
Such a useless grade should certainly not be used to declare a school a “failure” and in need of radical reform.
This is one variant of an ALEC sponsored scheme. Ohio has a similar scheme. It is certain to make all schools in four or five large metro areas candidates for state takeover, just like Youngstown. Thank you for pushing back on this. It needs your talent and that of your colleagues to disclose and publicize how this scheme is rigged to “fail” a high proportion of schools.
Most interesting…the article reminded me of how we, at the university level, have uneven grading schemes/scales for our own students. 90% might be an A in one class, but another professor might use a scale in which a 90% is a B+. While WE, as instructors, are usually evaluated as individuals, I can see how the issue of grading whole schools seems even MORE arbitrary than what we do as individuals. Each day, I learn a bit more about the dilemmas faced by K-12 teachers, schools, and districts.
A very nice read!
Followup with sources: In the latest highly reductive strategy for reporting school evaluations, some state officials have adopted model legislation from the American Legislative Exchange Council (2011). Teachers and schools are assigned letter grades, thereby obscuring a host of issues with the underlying VAMs and cut scores while appearing to use nothing more complex that a traditional A-to-F grading system. However, as many as nine performance indicators are graded and configured in a single rating. For example, a school cannot receive an “A” if any subgroup of students is awarded a “C.” Some grades are based on attaining a year’s worth of growth. It comes as no surprise that such grades mirror the SES profiles for communities (Amos & Brown, 2013).
American Legislative Exchange Council. (2011, January). A-Plus literacy act, Model legislation: Chapter 1. School and district report cards and grades. Retrieved from http://www.alec.org/model-legislation/the-a-plus-literacy-act/
Amos, D.S., & Brown, J. (2013, August 22). State unveils new report cards. Cincinnati Enquirer. Retrieved from http://news.cincinnati.com/article/20130822/NEWS0102/308220025/State-unveils-revamped-report-cards
Ohio’s evaluation system for public schools is rigged to fail many schools in urban districts. That makes the whole district a target for unelected charter operators, many with profit-seeking as the priority. In addition to Youngstown, the following districts are vulnerable to a district-wide takeover with charters becoming the only “choice”… and choosing students. Ohio just received $71 million from USDE to expand charters, up to 400. I read the evaluations of the application. They were useless in doing any triage on one of the states with the most notorious charter track records. Hanging on by a thread from becoming Youngstown:
Akron city schools , Canton city schools, Columbus city schools
Cleveland municipal schools, Cincinnati city schools, Toledo city schools
Dayton city schools
http://reportcard.education.ohio.gov/Pages/District-Report.aspx?DistrictIRN=043752