Following VAMboozled!’s most recent post (November 25, 2013) about the non-peer-reviewed National Bureau of Economic Research’s DC IMPACT study, another recently published study albeit this time in the Journal of Labor Economics, a top-field peer-reviewed journal, found no impact of incentives in New York City’s (NYC) public schools given its large-scale multimillion program. The author, Roland Fryer Jr., (2013) analyzed the data from a school-based experiment “to better understand the impact of teacher incentives on student achievement.”
A randomized experiment, a gold standard in applied work of this kind, was implemented in more than 200 hundred NYC public schools. The schools decided on the specific incentive scheme, either team or individual. The stakes were relatively high – on average, a high performing school (i.e. a school that meets the target by 100%), received a transfer of $180,000, and a school that met the target by 75%, received $90,000. Not bad by all accounts!
The target was set based on a school performance in terms of students’ achievement, improvement, and the learning environment. Yes, a fraction of schools met the target and received the transfers, but it did not improve the achievement of students, to say the least. If anything, such incentive in fact worsened the performance of students. The estimates from the experiment imply that if a student attended a middle school with an incentive in place for three years, his/her math test scores would decline by 0.138 of a standard deviation and his/her reading score would drop by 0.09 of a standard deviation.
Not only that, but the incentive program had no effect on teachers’ absenteeism, retention in school or district, nor did it affect the teachers’ perception of the learning environment in a school. Literally, the estimated 75 million dollars invested and spent brought zero return!
This evaluation, together with a few others (Glazerman & Seifullah, 2010; Springer et al., 2010; Vigdor, 2008) raises a question about both the financial effectiveness of similar incentives in schools and the achievement-based accountability measures in particular, and their ability to positively affect students’ achievement.
Thanks to Margarita Pivovarova – Assistant Professor of Economics at Arizona State University – for this post.
References:
Fryer, R. G. (2013). Teacher incentives and student achievement: Evidence from New York City Public Schools. Journal of Labor Economics, 31(2), 373-407.
Glazerman, S., & Seifullah, A. (2010). An evaluation of the Teacher Advancement Program (TAP) in Chicago: Year two impact report. Washington, DC: Mathematica Policy Research. Retrieved from http://www.mathematica-mpr.com/publications/pdfs/education/tap_yr2_rpt.pdf
Springer, M. G., Ballou, D., Hamilton, L. S., Le, V.-N., Lockwood, J.R., McCaffrey, D.F., Pepper, M., & Stecher, B.M. (2010). Teacher pay for performance: Experimental evidence from the project on incentives in teaching. Nashville, TN: National Center on Performance Incentives. Retrieved from http://www.rand.org/content/dam/rand/pubs/reprints/2010/RAND_RP1416.pdf
Vigdor, J. L. (2008). Teacher salary bonuses in North Carolina. Nashville, TN: National Center on Performance Incentives. Retrieved from https://my.vanderbilt.edu/performanceincentives/files/2012/10/200803_Vigdor_TeacherBonusesNC.pdf
