Skip to main content


Welcome, the Hub connects all projects

Voices From The Field


MSPnet Blog: “Value-added measures: Why all the love?”

See All Blog Posts

posted December 18, 2014 – by Brian Drayton

I really am curious: Why are policy makers, and some educators, so deeply convinced of the value of value-added measures (VAMs), or “student growth” measures, that they are willing to set them up as arbiters of teachers’ work and worth?   Maybe you can tell me why VAMs are getting such cred.
The question persists because the evidence of flaws in the system keeps mounting, and advice against reliance upon them except in the most guarded fashion keeps mounting as well.
I do understand the appeal of the basic logic, which can be cartooned sort of like this: Let’s model an input-output system. Measure a student score before she goes into the box, and measure the student’s score when she emerges, and predict what the score should have been. A big enough difference allows us to say the box did a good thing. A smaller difference justifies our saying that the box didn’t do such a good thing. If what’s in the box is a teacher, then we know what value that teacher added for that student. Repeat for other students passing through the same box. Voila! We know something about the teacher, who is the factor in the box!
Except it turns out we don’t know nearly as much as we think, and not nearly as reliably. The National Association of Secondary School Principals (NASSP) has published (here)  a very useful overview of the research, as part of their recommendation about how VAMs may and should not be used.  This follows a very thoughtful statement by the American Statistical Association (here).  The bottom line is that VAMs are unstable and unreliable from year to year, are subject to “noise” from a lot of different sources, and are best used as a small part of a more comprehensive and coherent program for teacher (and school) evaluation.

But as Daniel Katz discusses, the Dept of Education is advocating that VAM-type or student growth measures should be used to judge — not just teachers, but teacher preparation programs (see the announcement at the ED website). Katz provides links to a range of useful resources, and a compelling examination of VAM misapplication in the case  of a teacher in New York City. The research on VAMs and similar student growth measures makes clear that this anecdote is an illustration of an endemic problem, and is not merely cherry-picked sensation-mongering.

The ED confidence in, and advocacy of VAM (“or student growth”) metrics  seems to be adding new apartments to the air-castle of accountability measures that the accountability movement has been building, with enthusiastic support from many quarters, in defiance of the evidence, and at some considerable cost to our educational enterprise.

I have to say that as I read Katz’s post, followed his links, and revisited other studies I’d seen before, I was reminded of the argument of a favorite book (Voltaire’s Bastards:  the dictatorship of reason in the West, by J.R. Saul).  Saul argues that one of the unintended consequences of the Enlightenment was the rise of a cult of expertise, in which reasoned judgment is over-ruled by authorities whose metrics are protected from critique, even on the basis of the data that they claim to revere.  Education is not the only place where this problem can be espied, certainly;  but it’s not the least important sphere in which to struggle against facile but ill-founded policies.  There are real human consequences.

But maybe I am missing something.  What do you think?  Have you seen a way to square the circle,  and improve the models?   Have you been subjected to “growth models” that measured your impact on your students (or your teachers!), and did it feel like a truthful evaluation?


Blog comments have been archived, commenting is no longer available.
This blog post has 7 comments, showing all.

A different path

posted by: David Thomas on 12/28/2014 10:12 pm

So how does a school district blessed with enlightened leadership, teachers, and parents find and follow a different path than that prescribed by powerful political and business forces bent of treating all students and teachers as interchangeable machine parts?

Where is the dialogue happening?

posted by: Brian Drayton on 1/12/2015 11:16 am

I suspect that, as so often in education, there is a lot to be gained by the development of a convincing model. The debate about VAMs is taking place at various levels across the educational landscape, but it's happening in parallel with the debates about mandated testing, the Common Core, school funding, etc.
Are there places that have said, OK, the proponents of VAMs are trying to accomplish certain goals which we agree are valuable, but we have a different way of achieving them? Linda Darling-Hammond et al. (2011) explored this territory (see http://eml.berkeley.edu/~jrothst/publications/Evaluating%20Teacher%20E valuation%20-%20PDK%20prepublication.pdf ) --
Are you hearing conversations about alternatives as well as critiques?

post moderated by Kimberly Descoteaux on 1/14/2015

link

posted by: Julie Cantrell on 1/13/2015 7:45 am

Could you check the link? It may just be block by my school, but I am unable to read the article.

link

posted by: Kimberly Descoteaux on 1/14/2015 6:53 pm

The link should be fixed now.

VAM

posted by: Julie Cantrell on 1/13/2015 7:55 am

My district is completely misinterpreting VAM. My VAM scores comes from how my students do on a standardized test, compared to all of the other students in the district. It doesn't take into account where they start, just where they finish.
If we had a true VAM for my students, that information would be useful to me. If I could see trends across my students in different areas, I could improve my practice. As it is now, I don't have the starting point, only the end.

new commentary on VAM from Powell at Edweek

posted by: Brian Drayton on 3/10/2015 1:47 pm

People following this discussion might be interested by this blog post:
http://blogs.edweek.org/edweek/K-12_Contrarian/2015/03/the_declining_v alue_of_value-added_models.html

Arne Duncan and VAM, latest sightings

posted by: Brian Drayton on 4/17/2015 3:39 pm

With a recent issue of Educational Researcher coming out on VAMs, the MSPnet Blog will probably revisit this topic. Meantime, here is an interesting post from The Answer Sheet.
http://www.washingtonpost.com/blogs/answer-sheet/wp/2015/04/14/the-odd -thing-arne-duncan-told-congress/

This reports on a Q&A with Ed. Sec. Duncan, in which he retreats from the emphasis on test scores (and "growth models," which = VAM). His retreat, however, does not seem to be on principle, or on the science/math of the issue, but on the rhetoric. But read for yourself, and let us know what you think!