Litigating Algorithms, Beyond Education

This past June, I presented at a conference at New York University (NYU) called Litigating Algorithms. Most attendees were lawyers, law students, and the like, all of whom were there to discuss the multiple ways that they have collectively and independently been challenging governmental uses of algorithm-based, decision-making systems (i.e., like VAMs) across disciplines. I was there to present about how VAMs have been used by states and school districts in education, as well as present the key issues with VAMs as litigated via the lawsuits in which I have been engaged (e.g., Houston, New Mexico, New York, Tennessee, and Texas). The conference was sponsored by the AI Now Institute, also at NYU, which has as its mission to examine the social implications of artificial intelligence (AI), and in collaboration with the Center on Race, Inequality, and the Law, affiliated with the NYU School of Law.

Anyhow, they just released their report from this conference and I thought it important to share out with all of you, also in that it details the extent to which similar AI systems are being used across disciplines beyond education, and it details how such uses (misuses and abuses) are being litigated in court.

See the press release below, and see the full report here.

—–

Litigating Algorithms 2019 U.S. Report – New Challenges to Government Use of Algorithmic Decision Systems

Today the AI Now Institute and NYU Law’s Center on Race, Inequality, and the Law published new research on the ways litigation is being used as a tool to hold government accountable for using algorithmic tools that produce harmful results.

Algorithmic decision systems (ADS) are often sold as offering a number of benefits, from mitigating human bias and error, to cutting costs and increasing efficiency, accuracy, and reliability. Yet proof of these advantages is rarely offered, even as evidence of harm increases. Within health care, criminal justice, education, employment, and other areas, the implementation of these technologies has resulted in numerous problems with profound effects on millions of peoples’ lives.

More than 19,000 Michigan residents were incorrectly disqualified from food-assistance benefits by an errant ADS. A similar system automatically and arbitrarily cut Oregonians’ disability benefits. And an ADS falsely labeled 40,000 workers in Michigan as having committed unemployment fraud. These are a handful of examples that make clear the profound human consequences of the use of ADS, and the urgent need for accountability and validation mechanisms. 

In recent years, litigation has become a valuable tool for understanding the concrete and real impacts of flawed ADS and holding government accountable when it harms us. 

The Report picks up where our 2018 report left off, revisiting the first wave of U.S. lawsuits brought against government use of ADS, and examining what progress, if any, has been made.  We also explore a new wave of legal challenges that raise significant questions, including:

  1. What access, if any, criminal defense attorneys should have to law enforcement ADS in order to challenge allegations leveled by the prosecution; 
  2. The profound human consequences of erroneous or vindictive uses of governmental ADS; and 
  3. The evolution of the Illinois Biometric Information Privacy Act, America’s most powerful biometric privacy law, and what its potential impact on ADS accountability might be. 

This report offers concrete insights from actual cases involving plaintiffs and lawyers seeking justice in the face of harmful ADS. These cases illuminate many ways that ADS are perpetuating concrete harms, and the ways ADS companies are pushing against accountability and transparency.

The report also outlines several recommendations for advocates and other stakeholders interested in using litigation as a tool to hold government accountable for its use of ADS.

Citation: Richardson, R., Schultz, J. M., & Southerland, V. M. (2019). Litigating algorithms 2019 US report: New challenges to government use of algorithmic decision systems. New York, NY: AI Now Institute. Retrieved from https://ainowinstitute.org/litigatingalgorithms-2019-us.html

More on the VAM (Ab)Use in Florida

In my most recent post, about it being “Time to Organize in Florida” (see here), I wrote about how in Florida teachers were (two or so weeks after the start of the school year) being removed from teaching in Florida schools if their state-calculated, teacher-level VAM scores deemed them as teachers who “needed improvement” or were “unsatisfactory.” Yes – they were being removed from teaching in low performing schools IF their VAM scores, and VAM scores alone, deemed them as not adding value.

A reporter from the Tampa Bay Times with whom I spoke on this story just published his article all about this, titled “Florida’s ‘VAM Score’ for Rating Teachers is Still Around, and Still Hated” (see the full article here, and see the article’s full reference below). This piece captures the situation in Florida better than my prior post; hence, please give it a read.

Again, click here to read.

Full citation: Solochek, J. S. (2019). Florida’s ‘VAM score’ for rating teachers is still around, and still hated. Tampa Bay Times. Retrieved from https://www.tampabay.com/news/gradebook/2019/09/23/floridas-vam-score-for-rating-teachers-is-still-around-and-still-hated/

Time to Organize in Florida

A few weeks ago, a Florida reporter reached out to me for information about the nation’s value-added models (VAMs), but also as specific to the state of Florida. It seems that teachers in Florida were (and perhaps still are) being removed from teaching in Florida schools if their state-calculated, teacher-level VAM scores deemed them as teachers who “needed improvement” or were “unsatisfactory.”

More specifically, the state of Florida is using its state-level VAM to rate teachers’ VAM-based performance, using state exams in mathematics and language arts. If the teachers ultimately deemed in need of improvement or unsatisfactory teach in one of the state’s “turnaround” schools (i.e., a school that is required by the state to have a turnaround plan in place), those teachers are to be removed from the school and placed elsewhere. This is happening by state law, whereby the law dictates that no turnaround school may have a higher percentage of low value-added teachers than the district as a whole, which the state has apparently interpreted that to mean no low value-added teachers in these schools, at all.

Of course, some of the issues being raised throughout the state are not only about the VAMs themselves, as well as the teachers being displaced (e.g., two weeks or so after the school year resumed), but also about how all of this has caused other disruptions (e.g., students losing their teachers a few weeks after the beginning of the school year). Related, many principals have rejected these on-goings, expressly noting that they want to keep many if not most/all of the teachers being moved from their schools, as “valued” by them. I have also heard directly from a few Florida principals/school administrators about these same matters. See other articles about this here and here.

Hence, I’m writing this blog post to not only let others know about what is going on in Florida right now, despite the fact that the rest of the nation is (overall) taking some major steps back away from the uses (and abuses) of VAMs, especially in such high-stakes ways.

But I’m also writing this blog post to (hopefully) inspire those in Florida (including teachers, principals, etc.) to organize. Organize yourselves, perhaps with the assistance and guidance of your unions, professional organizations, legal groups (perhaps, also as affiliated), and the like. What is happening in Florida, as per state law, can very likely be legally challenged.

Overall, we (including many others in similar court cases in New Mexico, New York, and Texas) did quite well, overall, in the courts fighting the unjustifiable and indefensible uses of VAMs for similar purpose. Hence, I truly believe it is just a matter of time, with some organizing, that the teachers in Florida also realize some relief. There are also many of us out there who are more than ready and willing to help.