Category Archives: Video

Rethinking Value-Added Models (VAMs): A (Short) YouTube Version

Posted on by
Reply

Following up on a recent post about a recently released review of my book “Rethinking Value-Added Models in Education: Critical Perspectives on Tests and Assessment-Based Accountability,” I thought it important to share with you all a condensed, video- and cartoon-based version of the (very general) points highlighted within my book.

This YouTube video, also titled “Rethinking Value-Added Models in Education,” was created by one of my former doctoral students and one of the most amazing artists I know – Dr. Taryl Hargens.

Do give it a watch, and of course feel free to share out with others!!

A “Must Watch” Video on VAMs from Albuquerque, New Mexico

Posted on by
Reply

As written into a recent article in the Albuquerque Journal, an Albuquerque, New Mexico Public School Board member publicly, but in many ways appropriately, unleashed her frustration over the use of standardized tests and VAMs in Albuquerque’s public schools.

This is a “must watch” video that can be watched in just over two minutes here.

At one point she said that she is “so goddamn grateful she (her eldest of four children, a high school senior) is leaving the public schools” system at the end of the school year, although she still has three children left in the system.” Hence, her outrage, but also her call to her fellow board members to delay, or rather suspend indefinitely the use of the state of New Mexico’s new Common Core tests and to suspend the use of such test scores in teacher evaluations using VAMs, also suspect as per the board. In the state of New Mexico, test scores are to make up 50% of all test-eligible teacher’s evaluation scores. She also cited the American Statistical Association (ASA) recently released Position Statement on VAMs (discussed here and directly accessible here).

A fellow board member criticized Kort for her language choices that he defined as “poor.” He and a few other members also criticized her for not wanting to continue to work with the state that is pushing VAMs as working and negotiating with the state is in their best interests. Thereafter, “[t]he board passed a resolution calling on itself to draft a resolution addressing its concerns with the PARCC exam and its role in teacher evaluations and school grades. The board is looking to get feedback from parents and teachers before drafting its resolution.”

Vergara in New York, Thanks (in Part) to Campbell Brown

Posted on by
1

In a post I wrote about “Vergara Going on Tour,” I wrote about how the financier of the Vergara v. California case was preparing to bring similar suits to New York, Connecticut, Maryland, Oregon, New Mexico, Idaho, and Kansas. As well, the law firm that won the Vergara case for the plaintiffs, was also reported to have officially signed on to help defend the Houston Independent School District (HISD) on the forthcoming lawsuit during which, this time, the court will be investigating the EVAAS value-added system and its low- and high-stakes uses in HISD (this is also the source of a recent post here).

Last month, it was reported that New York was the next state on the tour, so-to-speak. To read a post from July about all of this, written by Bruce Baker at Rutgers titled “The VergarGuments are Coming to New York State!” click here.

It also seems that Campbell Brown, previous host of the Campbell Brown Show on CNN and award winning news anchor/journalist for multiple media outlets elsewhere, has joined “the cause” and even started her own foundation in support, aptly named the Partnership for Education Justice. Read more about their mission, as well as “The Challenge” and “The Solution” in America’s public schools as per America’s public school teachers as they define these here.

In New York specifically, via their first but unfortunately and likely not their last “project,” they are helping families “fight for the great teachers their children deserve by challenging factory-era laws that keep poorly-performing teachers in the classroom.” Read also about “The Problem,” the “Roadblocks,” and the like as they pertain to this specific suit in New York here. It probably won’t surprise you to see what research they are using to justify their advocacy work either – give it a quick guess and then check to verify here. Here is also a related article Brown recently wrote about how she (with all of her wisdom about America’s public school system – sorry) feels about teacher tenure.

Anyhow, last month (July 31, 2014) she was interviewed by Stephen Colbert on The Colbert Report on the Comedy Channel. Give this a watch to see what this is all about, in her terms and as per her (wrongheaded, misinformed, etc.) perspectives. See also Colbert’s funny but also wise response(s) to her aforementioned perspectives.

Watch it here:

 

 

The Colbert Report i

UCLA Professor Emeritus W. James Popham: His Testing and Teacher Evaluation Infomercial

Posted on by
4

W. James Popham, Emeritus Professor in the Graduate School of Education at the University of California, Los Angeles (UCLA), is best known for his decades of research on testing and assessment. Many of you might be familiar with his work have you ever read his classic textbook on assessment: Classroom Assessment: What Teachers Need to Know. 

He is also best known for having quite a sense of humor. Check out a new infomercial he produced about tests, tests as they relate to teacher evaluation (e.g., of the growth and value-added kind), and teacher observational systems as they too relate to teacher evaluation systems as currently being based on “multiple measures.”

The whole infomercial is just over 10 minutes, but do give it your full attention. There were parts I literally laughed out loud, although at the same time I could not get over the irony. Enjoy!

Please Refrain from “Think[ing] of VAMs Like an Oak Tree”

Posted on by
Reply

It happened again. In the Tampa Bay Times a journalist encouraged his readers to, as per the title of his article, “Think of VAMs Like an Oak Tree” as folks in Florida are now beginning to interpret and consume Florida teachers’ “value-added” data. It even seems that folks there are “pass[ing] around the University of Wisconsin’s ‘[O]ak [T]ree [A]nalogy,” to help others understand, unfortunately, what is a very over-simplistic and overoptimistic version of the very complex realities surrounding VAMs.

He, and others, obviously missed the memo.

So, I am redirecting current and future readers to Stanford Professor Edward Haertel’s deconstruction of the “Oak Tree Analogy,” so that we all might better spread the word about this faulty analogy.

I have also re-pasted Professor Haertel’s critique below:

“The Value-Added Research Center’s ‘Oak Tree’ analogy is helpful in conveying the theory [emphasis added] behind value-added models. To compare the two gardeners, we adjust away various influences that are out of the gardeners’ control, and then, as with value added, we just assume that whatever is left over must have been due to the gardener.  But, we can draw some important lessons from this analogy in addition to those highlighted in the presentation.

In the illustration, the overall effect of rainfall was an 8-inch difference in annual growth (+3 inches for one gardener’s location; -5 for the other). Effects of soil and temperature, in one direction or the other, were 5 inches and 13 inches. But the estimated effect of the gardeners themselves was only a 4-inch difference. 

As with teaching, the value-added model must sort out a small “signal” from a much larger amount of “noise” in estimating the effects of interest. It follows that the answer obtained may depend critically on just what influences are adjusted for. Why adjust for soil condition? Couldn’t a skillful gardener aerate the soil or amend it with fertilizer? If we adjust only for rainfall and temperature then Gardener B wins. If we add in the soil adjustment, then Gardener A wins. Teasing apart precisely those factors for which teachers justifiably should be held accountable versus those beyond their control may be well-nigh impossible, and if some adjustments are left out, the results will change. 

Another message comes from the focus on oak tree height as the outcome variable.  The savvy gardener might improve the height measure by removing lower limbs to force growth in just one direction, just as the savvy teacher might improve standardized test scores by focusing instruction narrowly on tested content. If there are stakes attached to these gardener comparisons, the oak trees may suffer.

The oak tree height analogy also highlights another point. Think about the problem of measuring the exact height of a tree—not a little sketch on a PowerPoint slide, but a real tree. How confidently could you say how tall it was to the nearest inch?  Where, exactly, would you put your tape measure? Would you measure to the topmost branch, the topmost twig, or the topmost leaf? On a sunny day, or at a time when the leaves and branches were heavy with rain?

The oak tree analogy does not discuss measurement error. But one of the most profound limitations of value-added models, when used for individual decision making, is their degree of error, referred to technically as low reliability. Simply put, if we compare the same two gardeners again next year, it’s anyone’s guess which of the two will come out ahead.”

A Tennessee Teacher, On the TVAAS and Other Issues of Concern

Posted on by
Reply

Check out this 5-minute video to hear from a teacher in Tennessee – the state recognized for bringing to the country value-added models and VAM-based teacher accountability – as she explains who things are going in her state of Tennessee.

Diane Ravitch, in her call to all of us to share out this and other videos/stories such as these, writes that we should help this video, now with over 100,000 views, reach every parent and teacher across the country. “We can be the change,” and social media can help us counter the nonsense expressed so well herein.

Comparing Oak Trees’ “Apples to Apples,” by Stanford’s Edward Haertel

Posted on by
2

A VAMboozled! follower posted this comment via Facebook the other day: “I was wondering if you had seen this video by The Value-Added Research Center [VARC], called the “Oak Tree Analogy” [it is the second video down]? My children’s school district has it on their web-site. What are your thoughts about VARC, and the video?”

I have my own thoughts about VARC, and I will share these next, but better than that I have somebody else’s much wiser thoughts about this video, as this video has in many ways gone “viral.”

Professor Edward Haertel, School of Education at Stanford University, wrote Linda Darling-Hammond (Stanford), Jesse Rothstein (Berkeley), and me an email a few years ago about just this video. While I could not find the email he eloquently drafted then, I persuaded (aka, begged) him to recreate what he wrote then, here, for all of you.

You might want to watch the video, first, to follow along, or least, to more critically view the contents of the video. You decide, but Professor Haertel writes:

“The Value-Added Research Center’s ‘Oak Tree’ analogy is helpful in conveying the theory [emphasis added] behind value-added models. To compare the two gardeners, we adjust away various influences that are out of the gardeners’ control, and then, as with value added, we just assume that whatever is left over must have been due to the gardener.  But, we can draw some important lessons from this analogy in addition to those highlighted in the presentation.

In the illustration, the overall effect of rainfall was an 8-inch difference in annual growth (+3 inches for one gardener’s location; -5 for the other). Effects of soil and temperature, in one direction or the other, were 5 inches and 13 inches. But the estimated effect of the gardeners themselves was only a 4-inch difference. 

As with teaching, the value-added model must sort out a small “signal” from a much larger amount of “noise” in estimating the effects of interest. It follows that the answer obtained may depend critically on just what influences are adjusted for. Why adjust for soil condition? Couldn’t a skillful gardener aerate the soil or amend it with fertilizer? If we adjust only for rainfall and temperature then Gardener B wins. If we add in the soil adjustment, then Gardener A wins. Teasing apart precisely those factors for which teachers justifiably should be held accountable versus those beyond their control may be well-nigh impossible, and if some adjustments are left out, the results will change. 

Another message comes from the focus on oak tree height as the outcome variable.  The savvy gardener might improve the height measure by removing lower limbs to force growth in just one direction, just as the savvy teacher might improve standardized test scores by focusing instruction narrowly on tested content. If there are stakes attached to these gardener comparisons, the oak trees may suffer.

The oak tree height analogy also highlights another point. Think about the problem of measuring the exact height of a tree—not a little sketch on a PowerPoint slide, but a real tree. How confidently could you say how tall it was to the nearest inch?  Where, exactly, would you put your tape measure? Would you measure to the topmost branch, the topmost twig, or the topmost leaf? On a sunny day, or at a time when the leaves and branches were heavy with rain?

The oak tree analogy does not discuss measurement error. But one of the most profound limitations of value-added models, when used for individual decision making, is their degree of error, referred to technically as low reliability. Simply put, if we compare the same two gardeners again next year, it’s anyone’s guess which of the two will come out ahead.”

Thanks are very much in order, Professor Haertel, for having “added value” to the conversations surrounding these issues, and, helping us collectively understand the not-so-simple theory advanced via this video.