| « ESEA Reauthorization in 2010? | NEA on Teacher Quality and Equity: Crossroads or Another Blind Alley? » |
Standardization is Not a Four-Letter Word
10/05/09
07:28:09 am, Categories: Announcements [A] Hat tip to Alexander Russo for posting this morning on an article in the L.A. Times about Douglas Reeves’ work on "subjective grading."
Here is a summary of Reeves’ work excerpted from the LAT:
"Reeves asked teachers and administrators in the United States, Australia, Canada and South America to determine a final semester grade for a student who received the following grades for assignments, in this order:
"C, C, MA (Missing Assignment), D, C, B, MA, MA, B, A.
"The educators gave the student final semester grades from A to F, Reeves said.
"Why? Because, he said, teachers use different criteria for grading."
Note that the L.A. Times doesn’t even get into the issue of the subjectivity that went into assigning each of the individual letter grades in the first place.
Imagine how biases about students affects the directionality of subjectivity. Do students who "act" smarter get the benefit of the doubt and get higher grades? Do teachers’ biases about poverty and race play a role? Do a students’ non-academic activities - whether or not he or she plays sports or is a behavior problem - tip the scales?
A large body of research says the answer to each of these questions is a resounding "yes."
This, in fact, is one reason why over the last 20 years we have come as a society to embrace "standardized" testing. But because most of the tests that are out there now are narrow and of poor quality, the term "standardized" has become a bad word, even though standardization is aimed at eliminating the types of biases reflected in Reeves’ work.
In attempts to make assessments work better, both for the purposes of evaluating individual students and school systems, it’s important that we not return to practices that rely on determining student achievement via a system in which the grade a student receives has more to do with the idiosyncratic grading biases of his or her teacher than with the student’s actual academic progress.
In the book "The Big Test" (2000) Nicholas Lemann grappled with the promise and shortcomings of testing in grading students based on their achievements, as opposed to assigning them college slots based on their family’s social status, as the U.S. did up until a half century ago, or by merely assigning post-secondary slots based on historical disadvantage, as is done through affirmative action. The book raises as many questions as it answers, but is nonetheless worth a re-read as we consider how testing under ESEA reauthorization can best serve the competing goals of maintaining fairness, rewarding success and hard work, and redressing historical education inequities.
Trackback address for this post
Trackback URL (right click and copy shortcut/link location)
2 comments
I would suggest the following when it comes to evaluating grading:
1. Research warns against assigning individual grades to individual assignments because averaging distorts accuracy.
2. The research you're looking at here is not nearly as thorough as the research in Robert Marzano's "Transforming Classroom Grading." Check it out. It's very short and very readable. And it covers 100 years of research on grading.
3. US grading expert Thomas Guskey says this: "Teachers do not need grades to teach. Students do not grades to learn. So we must come to some conclusion as to the real purpose of grading."
Finally, here's a research-based grading system that many people have used all over the country:
www.ttms.org/a_new_approach_to_grading_packet.pdf
This approach follows much of the work of Marzano and Guskey and limits the effects of subjectivity by multi-rater feedback.
Cheerfully,
Steve Peha
President, Teaching That Makes Sense
www.ttms.org
1. Research warns against assigning individual grades to individual assignments because averaging distorts accuracy.
2. The research you're looking at here is not nearly as thorough as the research in Robert Marzano's "Transforming Classroom Grading." Check it out. It's very short and very readable. And it covers 100 years of research on grading.
3. US grading expert Thomas Guskey says this: "Teachers do not need grades to teach. Students do not grades to learn. So we must come to some conclusion as to the real purpose of grading."
Finally, here's a research-based grading system that many people have used all over the country:
www.ttms.org/a_new_approach_to_grading_packet.pdf
This approach follows much of the work of Marzano and Guskey and limits the effects of subjectivity by multi-rater feedback.
Cheerfully,
Steve Peha
President, Teaching That Makes Sense
www.ttms.org
10/05/09 @ 14:28
Comment from: john thompson [Visitor] · http://www.thisweekineducation.com
I am not pro subjectivity. I am pro good subjectivity.
Seriously, exercises like Reeves and Lynn Canady’s are valuable in forcing teachers to look anew at their cherished grading systems and recognizing the infinite number of contingencies that none of us can adjust for. The purpose is to make sure that no teacher “ever sees grading in the same way again.” In other words, the purpose of the exercise is not necessarily to root out subjectivity, but to foster a more informed, more open-minded, and more just subjectivity. Too many teachers are just as wedded to their zeros and formulas as you are to standardization. If standardization is a lousy idea (and I think it’s the worst of all possible worlds) the Reeves and Canady surveys do not add any credence to it. In fact, the call for mindful and meaningful grading could actually increase subjectivity by puncturing teachers’ most deeply held ideologies.
That is why attending a lecture by Canady (I don’t know Reeves) is doubly valuable. I don’t care what the subject is. Attend a lecture by Canady on any subject and you walk out seeing the world with new eyes.
Regarding your previous post, you may not believe it but I’m just as skeptical of localism as you are. But this may be a place where the unions understand the value of localism. Take for instance the issue of forced transfers. Other than a very few administrators in the biggest districts I can’t understand why reformers still want that issue and don’t explicitly repudiate it. Force a transfer of an effective teacher from an effective school to a failing school and in almost every case you will damage both schools. That would be doubly true of a place trying to implement what Arlene Ackerman claims she is trying to do. Transfer an unwilling teacher into a team that wants to turnaround a school and almost certainly you’ve poisoned the well.
I suspect the idea of forced transfers haven’t been explicitly repudiated because this is a classic area where the interests of the administrator and that of the district are contradictory. The district won’t benefit, but an individual administrator who is stymied in pushing his or her agenda wants to keep that option open for the benefit of his or own career.
The trickier issue is layoffs and turnarounds. As you know, I’m not opposed to negotiating provisions that are not dis-similar to layoffs and bankruptcy in industry. The problem is the issue of good faith. My problem with Duncan’s turnaround guidelines is not that he keeps all options open but that he favors the most disruptive approaches that could be the most damaging to unions. Rip up collective bargaining agreements in 2000 schools and you can get to the point where the expectation in the rule of law is the exception, not the rule.
The worst scenario is D.C. I don’t pretend to know the law in that unique situation. But did the editors of the Post ask questions about the legality of Rhee layoff procedures? They implied they’d support Rhee regardless.
So getting back to localism, local unions ought to be able to waive provisions where they see credible opportunities to achieve equity while protecting the integrity of contracts. But no national union is going to ask their local to give the benefit of the doubt to someone like Rhee who puts her own morality over the rule of law.
And that gets us back to subjectivity. Subjectivity in grading may have annoyed us, but we weren’t hurt by it. The problem comes with the new challenge of educating kids who have been excluded historically. To address this challenge where none of us has solutions, we need more handshake agreements not less. We need more good-faith collaboration not more coercion. We need more mindful inquiry and less mindless standardization and top-down directives.
Seriously, exercises like Reeves and Lynn Canady’s are valuable in forcing teachers to look anew at their cherished grading systems and recognizing the infinite number of contingencies that none of us can adjust for. The purpose is to make sure that no teacher “ever sees grading in the same way again.” In other words, the purpose of the exercise is not necessarily to root out subjectivity, but to foster a more informed, more open-minded, and more just subjectivity. Too many teachers are just as wedded to their zeros and formulas as you are to standardization. If standardization is a lousy idea (and I think it’s the worst of all possible worlds) the Reeves and Canady surveys do not add any credence to it. In fact, the call for mindful and meaningful grading could actually increase subjectivity by puncturing teachers’ most deeply held ideologies.
That is why attending a lecture by Canady (I don’t know Reeves) is doubly valuable. I don’t care what the subject is. Attend a lecture by Canady on any subject and you walk out seeing the world with new eyes.
Regarding your previous post, you may not believe it but I’m just as skeptical of localism as you are. But this may be a place where the unions understand the value of localism. Take for instance the issue of forced transfers. Other than a very few administrators in the biggest districts I can’t understand why reformers still want that issue and don’t explicitly repudiate it. Force a transfer of an effective teacher from an effective school to a failing school and in almost every case you will damage both schools. That would be doubly true of a place trying to implement what Arlene Ackerman claims she is trying to do. Transfer an unwilling teacher into a team that wants to turnaround a school and almost certainly you’ve poisoned the well.
I suspect the idea of forced transfers haven’t been explicitly repudiated because this is a classic area where the interests of the administrator and that of the district are contradictory. The district won’t benefit, but an individual administrator who is stymied in pushing his or her agenda wants to keep that option open for the benefit of his or own career.
The trickier issue is layoffs and turnarounds. As you know, I’m not opposed to negotiating provisions that are not dis-similar to layoffs and bankruptcy in industry. The problem is the issue of good faith. My problem with Duncan’s turnaround guidelines is not that he keeps all options open but that he favors the most disruptive approaches that could be the most damaging to unions. Rip up collective bargaining agreements in 2000 schools and you can get to the point where the expectation in the rule of law is the exception, not the rule.
The worst scenario is D.C. I don’t pretend to know the law in that unique situation. But did the editors of the Post ask questions about the legality of Rhee layoff procedures? They implied they’d support Rhee regardless.
So getting back to localism, local unions ought to be able to waive provisions where they see credible opportunities to achieve equity while protecting the integrity of contracts. But no national union is going to ask their local to give the benefit of the doubt to someone like Rhee who puts her own morality over the rule of law.
And that gets us back to subjectivity. Subjectivity in grading may have annoyed us, but we weren’t hurt by it. The problem comes with the new challenge of educating kids who have been excluded historically. To address this challenge where none of us has solutions, we need more handshake agreements not less. We need more good-faith collaboration not more coercion. We need more mindful inquiry and less mindless standardization and top-down directives.
10/05/09 @ 15:30
This post has 854 feedbacks awaiting moderation...
Comment feed for this postLeave a comment
