A “Must Watch” Video on VAMs from Albuquerque, New Mexico

Share Button

As written into a recent article in the Albuquerque Journal, an Albuquerque, New Mexico Public School Board member publicly, but in many ways appropriately, unleashed her frustration over the use of standardized tests and VAMs in Albuquerque’s public schools.

This is a “must watch” video that can be watched in just over two minutes here.

At one point she said that she is “so goddamn grateful she (her eldest of four children, a high school senior) is leaving the public schools” system at the end of the school year, although she still has three children left in the system.” Hence, her outrage, but also her call to her fellow board members to delay, or rather suspend indefinitely the use of the state of New Mexico’s new Common Core tests and to suspend the use of such test scores in teacher evaluations using VAMs, also suspect as per the board. In the state of New Mexico, test scores are to make up 50% of all test-eligible teacher’s evaluation scores. She also cited the American Statistical Association (ASA) recently released Position Statement on VAMs (discussed here and directly accessible here).

A fellow board member criticized Kort for her language choices that he defined as “poor.” He and a few other members also criticized her for not wanting to continue to work with the state that is pushing VAMs as working and negotiating with the state is in their best interests. Thereafter, “[t]he board passed a resolution calling on itself to draft a resolution addressing its concerns with the PARCC exam and its role in teacher evaluations and school grades. The board is looking to get feedback from parents and teachers before drafting its resolution.”

Share Button

VAMboozled! One Year Later

Share Button

This VAMboozled blog officially went live just over one year ago two weeks ago. You can read our first post, here, about the “Top 10″ pieces of VAMmunition, or rather, ammunition, practitioners need to know to better protect themselves against the unfair implementation and use of VAMs. This one is definitely worth another read if you haven’t read it before as it captures, what I still believe are the same  research-based reasons, today, that VAMs don’t work as intended and as per the research. To read this list, click again here.

Otherwise, I thought I’d share with you all our “progress” report as per our Google analytics. We are now up to ≈ 8,200 subscribers with a loose approximate of about 100,000* hits or blog reads per month.

Hence, thank you all for your support, interest, sharing, action, and the like!! Onward!! And here’s to a productive year in terms of informing more about the research and research-based issues surrounding teacher evaluation, teacher accountability, and value-added models (VAMs) in America’s public schools.

*This number is calculated by ((subscribers x average number of posts per month) + external hits per month)), although external hits may also include subscribers as analytics cannot differentiate between subscribers and hits. These two indicators are not mutually exclusive.

Share Button

Breaking News: Tennessee’s Huffman and Tulane’s Ayers Resign

Share Button

Two public faces in the world of VAM announced this week that they are resigning from their leadership positions, so it is being claimed, given their high visibility (and perhaps failures) using VAMs.

First, Kevin Huffman, a big national champion of teacher accountability and perpetuator of unsubstantiated nonsense about how teachers need to “work harder” and be held more accountable when they don’t do so, just resigned from his position as the Tennessee Education Commissioner. As per an article in The Tennessean: “Huffman — a former Teach For America executive and ex-husband of controversial former Washington, D.C., Chancellor Michelle Rhee — made Tennessee a flashpoint nationally for debates over teacher evaluations, teacher licensing, Common Core standards and charter schools…In recent months, Huffman lost a handful of political fights. The state overturned a teacher license policy that tied test scores to advancing professionally and delayed a move to Common Core-aligned state testing…More than 50 superintendents had already publicly questioned his leadership, several teachers unions expressed “no confidence” even after [other] accolades; and most recently a group of 15 Republicans last summer called for his resignation.” See previous VAMboozled posts about Huffman’s more-than-controversial past in the state of Tennessee here, here, and here, but more importantly click here for the full article (from which I pulled excerpts above) explaining Huffman’s past and resignation.

Second, John Ayers, the executive director of Tulane University’s Cowen Institute for Education Initiatives, announced that he is to resign at the end of November in an article just posted in the npr_ED blog. One month ago, in what was being called a “high-profile embarrassment,” Ayers on behalf of the institute apologized for releasing a high-profile report based on faulty VAM research and then pulled the institute’s report from its once prevalent placement on its website. The report was celebrated widely in that it “proved” that students in post-Hurricane Katrina’s (largely charter) schools, despite the disadvantages they faced prior to the school-based reforms triggered by Katrina, were “beating the odds and “posting better test scores and graduation rates than predicted by their populations,” thanks to said reforms. Demographics were no longer predicting students’ educational destinies, and the institute had the VAM-based evidence to prove it. Let’s just say the leaders of the charter movement were all over this one, and also allegedly involved. It seems, now, this has at least in part led to the resignation of the institute’s leader, although “Ayers told the New Orleans Times-Picayune that the flawed report and his sudden departure are unrelated.” See a previous VAMboozled post about Ayer and this debacle here, but more importantly click here for the full nprED article in which the article author also writes quite eloquently about how “value-added modeling, it turns out, is really, really hard.”

Both incidents should “serve as a caution both to education researchers, and to those of us [e.g., policymakers, think tank organizations, members of the public] who are making decisions based on [VAM] findings.”

Share Button

More from Moshe on Chetty, Kane, and their Vergara Testimonies

Share Button

Moshe Adler is one of Chetty et al.’s biggest and most important critics, especially as an expert in economics and econometrics himself. Likewise, he has been debunking Chetty et al.’s works from the time Chetty et al.’s studies were first released in 2011 (when the studies were cited by by President Obama) through their most current 2014 versions (when the studies were cited by the judge in Vergara v. California). Examples of some examples of Adler’s criticisms of Chetty et al.’s works can be found here and here.

Adler is also a critic of the work of Thomas Kane, another person of interest (see prior posts about Kane here and here), as Kane is also involved with the Bill & Melinda Gates Foundation, he is a proponent of VAMs as aligned with the Gates Foundation’s interests, and he (like Chetty) served as a key witness for the prosecution in the same Vergara v. California trial. In many ways, Chetty and Kane together delivered the final blows that cost the defense its lost in the Vergara v. California case (as explained here).

That being said, Adler recently presented his work at a conference of the National Organization of Lawyers for Education Associations that was held in Washington DC in October of 2014. More specifically, he presented his review of Chetty’s and Kane’s Vergara v. California testimonies. He also agreed to share his PowerPoint presentation with a brief summary, below. Here’s what he wrote, although you can also click here for his full PowerPoint presentation.

In the Vergara trial in Los Angeles, the economist Raj Chetty testified that teachers with high teacher value-added scores have a life-long positive impact on the incomes of their students, and the economist Tom Kane testified that minority students in Los-Angeles lose more than a month of schooling each year because they are assigned low value-added teachers.

Chetty’s testimony is based on two articles that he co-authored with economists John Friedman and Jonah Rockoff. Tom Kane’s testimony is based on his work for the Bill and Melinda Gates Foundation.

However, what is highlighted in this PowerPoint presentation is that Chetty’s research, on which he based his testimony, constitutes bad science, and that Kane chose to present his findings in units that present trivial effects as significant [emphasis added].

When Chetty et al.’s articles were accepted for publication by the American Economic Review (AER), Chetty emailed me to inform me of this news (and to suggest that these articles should earn him the Nobel Prize). I immediately wrote to the editors of the AER to alert them of the problems with the articles. Curiously, the editors did not send my concerns to a third party reviewer or even to the referees who originally reviewed the papers for the review. Instead, they sent them to Chetty et al., who published a reply to which I have responded. For more information about all of this, see this aforementioned debate here, and see all of this email correspondence here.

But as for this PowerPoint, the main point is that Chetty et al.’s claim that an increase in teacher value-added leads to higher income throughout adulthood is contradicted by their own findings. Again, you can read more about this here.

Share Button

Rothstein, Chetty et al., and (Now) Kane on Bias

Share Button

Here’s an update to a recent post about research conducted by Berkeley Associate Professor of Economics – Jesse Rothstein.

In Rothstein’s recently released study, he provides evidence that puts the aforementioned Chetty et al. results under a more appropriate light. Rothstein’s charge, again, is that Chetty et al. (perhaps unintentionally) masked evidence of bias in their now infamous VAM-based study, which in turn biased Chetty et al.’s (perpetual) claims that teachers caused effects in student achievement growth over time. These effects, rather, might have been more likely caused by bias given the types of students non-randomly assigned to teachers’ classrooms versus “true teacher effects.”

In addition, while in his study Rothstein replicated Chetty et al.’s overall results using a similar data set, so did Thomas Kane – a colleague of Chetty’s at Harvard who has also been the source of prior VAMboozled! posts here, here, and here. During the Vergara v. California case last summer, the prosecuting team actually used Kane’s (and colleagues’) replication-study results to validate Chetty et al.’s initial results.

However, Rothstein did not replicate Chetty et al.’s findings when it came to bias (the best evidence of this is offered in Rothstein’s study’s Appendix B). Inversely, Kane’s (and colleagues’) study did not, then, have any of the prior year score analyses needed to analyze and assess bias, so the extent to which Chetty et al.’s results were due to bias was then more or less moot.

But after Rothstein released his recent study effectively critiquing Chetty et al. on this point, Kane (and colleagues) released the results Kane presented at the Vergara trial (see here). However, Kane (and colleagues) seemingly released an updated version of “Kane’s” initial results to seemingly counter Rothstein, in support of Chetty. In other words, Kane seems to have released his study (perhaps) more in support of his colleague Chetty than in the name of conducting good, independent research.

Oh the tangled web Chetty and Kane (purportedly) continue to weave.

See also Chetty et al.’s direct response to Rothstein here.

Share Button

Recent Gallup Poll on Teacher Perceptions

Share Button

In an “Update” published by the Gallup public opinion polling company titled “Teachers Favor Common Core Standards, Not the Testing,” some interesting data can be found regarding current teachers’ perceptions about some currently hot reform topics in education. These Gallup data include, as pertinent here, self-report data on how teachers feel about using students’ test scores to hold them accountable.

Here are what I see as the most interesting findings, again as pertinent here. For more information, however, do click here.

  • 76% of America’s public school teachers “reacted positively” to the primary goal of the Common Core State Standards (i.e., to have all states use the same set of academic standards for reading, writing and math in grades K-12).
  • 9% of America’s public school teachers were in favor of linking students’ test scores to teacher evaluations.
  • 89% of America’s public school teachers viewed linking students’ test scores to teacher evaluations negatively, and 2% reported having “no opinion” on the matter.
  • 78% of America’s public school teachers agreed that the tests to be used in general and also to hold them accountable for their effectiveness (e.g., using VAMs) take too much time away from teaching.

As one teacher who helped Gallup to interpret these findings put it: “The [Common Core] standards were positive until standardized testing was involved.”

As for linking students’ test scores to teacher performance, another teacher elaborated on this by stating, “Student populations are not spread evenly among teachers. Some students will have learning disabilities and some will have familial and environmental factors that do not allow them to progress at the same rate as others. If I have to be so concerned about a test score, I cannot address the needs of individual students.” And a Texas high school teacher concludes, “We must have different yardsticks for different student circumstances so teachers can focus on individual student growth.”

The teachers who participated in this survey research study included a random sample of 854 public K-12 school teachers, aged 18 and older, with Internet access, living in all 50 U.S. states and the District of Columbia.

Share Button

Education-Related Election Updates

Share Button

Great news! The state of Missouri did not pass its “VAM with a Vengeance” constitutional amendment proposing to tie teacher evaluation to student test scores and require teachers to be dismissed, retained, promoted, demoted, and paid based primarily on the test scores of their students. It was also to require teachers to enter into contracts of three years or less, eliminating seniority and tenure. Well done, voters of Missouri!!

Uncertain news! My state of Arizona is still counting ballots to determine who will be our new State Superintendent of Public Instruction. For the first time in history, we have somebody who actually knows a lot about education running – Dr. David Garcia – an associate professor colleague of mine at Arizona State University who also studies educational policy. He would also be the first Latino elected to a state office in over 30 years. He is running as a Democrat, though, and in a very red state, a win for him and others before him have often turned impossible. His opponent is Diane Douglas, a former school board member who focused everything about her campaign in protest of the Common Core Standards and the federal government’s involvement in state-level educational policies. She ran a low-key campaign, she dodged the media spotlight, and she avoided all but one debate with David during which David literally ate her lunch. BUT lucky for her, she ran on the Republican ticket. While there are still 300,000 votes to be counted, she has a narrow lead which is making many on both sides of the party line very upset with those in our state who perpetually vote along the party line. Even with David drawing some very significant endorsements from Republicans and conservative business groups throughout this red state, the state’s conservative voters continue to dominate. Many agree on this note and “see her success as part of the decisive Republican sweep and not a triumph of ideology.” So very sad for our state is this one here. The next move might just be amend the state constitution and make this a state appointed position in that David is by far the best and most informed candidate our state has likely every had in the running (anonymous phone conversation, this morning ;).

Unfortunate news! As per a recent post by Diane Ravitch, the other election news as related to education was bad. There were many victories for similar people “who hold the public sector in contempt and believe in Social Darwinism. It was a bad night for those who hope for a larger vision of the common good, some vision grander than each one on his own. American history and politics are cyclical. It may require the excesses of this time to bring a turn of the wheel. It’s always darkest just before dawn.”

Share Button

“VAM with a Vengeance” on the Ballot in Missouri

Share Button

On this election day, the state of Missouri is worth mentioning as it has a very relevant constitutional amendment on its ballot for the day. As featured on Diane Ravitch’s blog a few days ago here, this (see below) is “the worst constitutional amendment to appear on any state ballot in 2014.”

missouriballotissue

“It ties teacher evaluation to student test scores. It bans collective bargaining about teacher evaluation. It requires teachers to be dismissed, retained, promoted, demoted, and paid based primarily on the test scores of their students. It requires teachers to enter into contracts of three years or less, thus eliminating seniority and tenure.

This is VAM with a vengeance.

This ballot resolution is the work of the far-right Show-Me Institute, funded by the multi-millionaire Rex Sinquefeld. He is a major contributor to politics in Missouri and to ALEC.

The Center for Media and Democracy writes about him:

‘Sinquefield is doing to Missouri what the Koch Brothers are doing to the entire country. For the Koch Brothers and Sinquefield, a lot of the action these days is not at the national but at the state level.’

‘By examining what Sinquefield is up to in Missouri, you get a sobering glimpse of how the wealthiest conservatives are conducting a low-profile campaign to destroy civil society.’

‘Sinquefield told The Wall Street Journal in 2012 that his two main interests are “rolling back taxes” and “rescuing education from teachers’ unions.’

‘His anti-tax, anti-labor, and anti-public education views are common fare on the right. But what sets Sinquefield apart is the systematic way he has used his millions to try to push his private agenda down the throats of the citizens of Missouri.”

Share Button

“Highly Regarded” Teacher Suing New York over “Ineffective” VAM Scores

Share Button

According to a recent Washington [Blog] Post written by Valerie Strauss, in New York a 17-year veteran and current 4th grade teacher, recognized by the district superintendent as having a “flawless” teaching record and being “highly regarded as an educator,” is suing the state of New York over her VAM scores that have just placed her in the “ineffective” teacher category. This is, and will prove to be very important for her and perhaps other teachers’ in general, or who might also engage in similar legal actions elsewhere.

In this case, though, the teacher’s “students consistently outperform state averages on math and English standardized tests,” and the superintendent signed an affidavit on her behalf saying “her record is flawless” and that “she is highly regarded as an educator.” In addition, he noted “[her] classroom observations have consistently identified her as an exceptional educator,” among other praises.

So, “[h]ow is it that a teacher known for excellence could be rated ‘ineffective’?” One guess: VAM.

“[T]he New York State Growth Measures [i.e., New York’s version of VAM] ‘actually punishes excellence in education through a statistical black box which no rational educator or fact finder could see as fair, accurate or reliable.” For example, “[i]n 2012-13, 68.75 percent of her students met or exceeded state standards in both English and math. She was labeled “effective” that year. In 2013-14, her students’ test results were very similar but she was rated “ineffective.” This likely occurred because she did not “grow” her students as much the second year as compared to her comparison group of peers, although any system that can be used to rate the same teacher “effective” one year and “ineffective” the next should raise eyebrows, naturally. As per the lawsuit, “This simply [and obviously] makes no sense, both as a matter of statistics and as a matter of rating teachers based upon slight changes in student performance from year to year.”

This is certainly one to watch, and I look forward to other teachers following her lead as this is something likely to ultimately be won in the courthouse. Onward!

Share Button

Recommended Reading — 50 Myths and Lies that Threaten America’s Public Schools: The Real Crisis in Education

Share Button

A few months ago, two of the most renowned scholars in the field of education—Drs. David Berliner and Gene Glass, who are both mentors of mine here at Arizona State — wrote a book that is sure to stir up some engaging dialogue regarding public education. The two teamed up with a group of our PhD students as well as PhD students from the University of Colorado-Boulder in order to tackle what they have deemed the 50 Myths and Lies that Threaten America’s Public Schools: The Real Crisis in Education. While the book covers a wide range of topics including everything from charter schools, to bullying, to English acquisition programs, and even sex education, there are several chapters (or myths) that would likely be of particular interest to the readers of this blog (i.e., on teacher accountability and VAMs — see post forthcoming).

Otherwise, eight of the other myths, specifically, deal directly with the lies often told about teachers, including the way in which they should be evaluated based on student test scores (via VAMs). They include the following myths, again, deconstructed in this book:

  • Teachers are the most important influence in a child’s education.
  • Teachers in the United States are well-paid.
  • Merit pay is a good way to increase the performance of teachers. Teachers should be evaluated on the basis of the performance of their students. Rewarding and punishing schools for the performance of their students will improve our nation’s schools. – See forthcoming post about this myth specifically
  • Teachers in schools that serve the poor are not very talented.
  • Teach for America teachers are well trained, highly qualified, and get amazing results.
  • Subject matter knowledge is the most important asset a teacher can possess.
  • Teachers’ unions are responsible for much poor school performance. Incompetent teachers cannot be fired if they have tenure.
  • Judging teacher education programs by means of the scores that their teachers’ students get on state tests is a good way to judge the quality of the teacher education program.

At a time when corporate reformists are more interested in making money off of education than actually attempting to improve educational quality for all students, books like this are critical in helping us decipher the truth from the perpetual myths that have become the bedrock of the reform movement. Berliner and Glass do not hold back – they name names where possible and provide enough research to support their claims… but not too much to slow them down. Instead, they call upon decades’ worth of educational research, the trusty work of their students, and a bit of logic and humor in order to pack a powerful punch against those most responsible for spreading the myths and, often, flat-out lies about America’s students, teachers, and schools. This book is for teachers, parents, policymakers, school administrators, and concerned citizens alike. I definitely recommend that you add it to your reading list!

Share Button