Race to the bottom: Red Grader, Blue Grader

[Dreadnaught]

Red Grader, Blue Grader
May 20, 2011
Republican professors and Democratic professors presumably produce different outcomes when they enter the ballot box, but what about when they record grades?

A forthcoming study finds that there may be notable differences. Democratic professors appear to be "more egalitarian" than their Republican counterparts when it comes to grading, meaning that more of the Democratic grades are in the middle. Republicans are more likely than Democrats to award very high grades and very low grades.

Another key difference is that black students tend to fare better with Democrats than with Republicans.

While the study documents those differences, the work will not satisfy political partisans hoping to demonstrate that Republicans are trying to encourage Darwinian competition with grading or that Democrats are Lake Wobegon graders afraid to suggest anyone did poorly. That's because the study makes clear that the researchers lacked the information to determine whether the Democratic or Republican grades were better reflections of student performance. The only thing the researchers could vouch for was the politically linked pattern in grading.

The study -- forthcoming in Applied Economics -- is by Talia Bar, an assistant professor at Cornell University, and Asaf Zussman, assistant professor of economics at Hebrew University of Jerusalem. They examined thousands of grades in a dataset covering the grades awarded at an unnamed elite American university between 2000 and 2004. Party registrations were used to identify professors' political inclinations, and the faculty at this university leaned Democratic, especially among humanities professors. Using SAT scores as a proxy for the preparedness of students, the researchers were able to rule out patterns in which Republican or Democratic professors had better students.

On grade distribution, Republicans were more likely to give very high and very low grades. Among grades given by Republicans, 6.2 percent were C- or lower, compared to only 4.0 percent of the Democratic grades. But Republicans were also more likely to give out A+ grades (8 percent of their grades, compared to only 3.5 percent from Democrats).

With regard to race of students, the study found that black students received lower grades, on average, than did white students whether classes were taught by Democrats or Republicans. In courses with Democratic professors, the gap was 0.27 on a grade point average. In Republican-led classrooms, the gap was 0.42.

While the authors don't endorse the grading style of either Democrats or Republicans, they conclude their paper by arguing that people (such as those concerned about grade distribution) seeking to influence grading policies should pay more attention to the political factor. "Our results suggest that the allocation of grades is associated with the world view or ideology of professors," the authors write.

— Scott Jaschik

http://www.insidehighered.com/news/2...ading_patterns

Nothing useful will come out of this besides perhaps endless debates. Though I find it interesting that self-identified Republicans grade people with less "room in the middle".

[Loki]

The substantive effects look pretty small. The A+ and C-/lower grades only cover 10-12% of all the students. That means that 88-90% of the grades are not significantly different.

[Khendraja'aro]

Grades are useless subjective. Film at 11.

In regards to the article: It's called "Tendency towards extremes" and "Tendency towards the middle".

[Aimless]

So what you guys are saying is that you give this study C-?

I would be interested in learning whether or not they controlled for the classes being taught and the overall grading policy for those classes (ie. differences between eg. gender theory and econ, and differences in how the grades were portioned out eg. between essays, tests, etc). Other than that, meh

[Lewkowski]

Well I might be biased but its unlikely to me that everyone would be lumped in the middle. There will be over achievers and there will be under achievers. To me it sounds like the Republican method of grading is a more accurate reflection of the student body.

And wow what a lazy study. I mean come on give us some metric that we can determine which is more accurate. Post college employment? A standardized test they could give? Income level after college? Compare to non-ideological professors?

[Loki]

You're assuming any of those factors are able to predict grades with any accuracy. From my limited experience, the best predictor of a good grade is interest in the subject, though some people are naturally better at academic activities.

[LittleFuzzy]

Well I might be biased but its unlikely to me that everyone would be lumped in the middle. There will be over achievers and there will be under achievers. To me it sounds like the Republican method of grading is a more accurate reflection of the student body.

How? It's not that the Democratic professors didn't give you Fs and A+s, they just gave out less of them in proportion to the total. Of course there will be over-achievers, and under-achievers, and achievers sitting in the middle, and both clusters scoring reflect this. They simply have some differences in the total proportions *and since the statistics provided here were entirely relative, we don't even know what those proportions are, just how they differed between the groups*

[Lewkowski]

You're assuming any of those factors are able to predict grades with any accuracy. From my limited experience, the best predictor of a good grade is interest in the subject, though some people are naturally better at academic activities.

Shrug. I don't have any data to support it but I would assume there would be some correlation between obtaining good grades and getting a job.

[Loki]

Shrug. I don't have any data to support it but I would assume there would be some correlation between obtaining good grades and getting a job.

As long as a student gets at least a B average, very few employers are going to care. Seeing that the median grade in most colleges is around a B+, that means that well over 50% of the students are going to be getting the same jobs.

[Lewkowski]

As long as a student gets at least a B average, very few employers are going to care. Seeing that the median grade in most colleges is around a B+, that means that well over 50% of the students are going to be getting the same jobs.

I'm not saying the employer is going to look at the grades necessarily but that someone who gets high marks in class may be more motivated then someone else. Data could be looked at and I would make the case that probably the higher the GPA (when controlled for class difficulty) would be positively correlated with job success.

[Loki]

I'm not saying the employer is going to look at the grades necessarily but that someone who gets high marks in class may be more motivated then someone else. Data could be looked at and I would make the case that probably the higher the GPA (when controlled for class difficulty) would be positively correlated with job success.

Don't you think employers would care more about the grades if such a correlation existed?

[Khendraja'aro]

In Germany we have the tendency that the social sciences usually have higher grading averages than the natural sciences (with Physics being the worst).

What would that mean in Lewk's world?

[Loki]

To be fair, people in the social sciences are generally not competing for the same jobs as people from the hard sciences.

[Khendraja'aro]

Yeah, but it's one more piece of evidence that grades are not exactly reliable.

[Loki]

They're reliable in the way they're used: if you can't manage a B average, chances are you're severely lacking in some important skill. Beyond that, they're not even particularly useful at predicting grad school success.

[Khendraja'aro]

That could also be achieved by a mere binary grade - fail/non-fail. Don't see why you need grades with 5 or 6 increments for that.

[Loki]

That could also be achieved by a mere binary grade - fail/non-fail. Don't see why you need grades with 5 or 6 increments for that.

Because people won't be happy to get a fail instead of a B-. As things stand, perhaps 5% get an F.

[Lewkowski]

That could also be achieved by a mere binary grade - fail/non-fail. Don't see why you need grades with 5 or 6 increments for that.

Its always a good idea to rank people. For example when its time for annual performance evaluations you are given a "grade" not a pass / fail. Most professional jobs have this sort of thing as opposed to "pass/fail." Why should college be different?

[Khendraja'aro]

Because people won't be happy to get a fail instead of a B-. As things stand, perhaps 5% get an F.

Since when is grading about what people are happy with?

Its always a good idea to rank people. For example when its time for annual performance evaluations you are given a "grade" not a pass / fail. Most professional jobs have this sort of thing as opposed to "pass/fail." Why should college be different?

Because grades are not the analytical instrument you make them out to be?

Listen, you can measure the velocity of a car. And even there people are going: "Oh, the instrument was not calibrated!", "The office got it wrong!" and so on and so forth. And that's a simple measurement.

And then we have grades where literally(!) everything under the moon comes into play. And you think that is reliable? Heh. You're even more naive than I thought. And you're the same person who's doubtful of climatology because "it's all numbers and the wrong ones to boot"?

[Aimless]

Didn't we have a discussion a while back about evaluating performance at a workplace, something about how eg. basing salaries on evaluations of "performance" influenced performance...? Didn't we also have a discussion about things like pass/fail systems eg. at med-schools? I don't remember

Since when is grading about what people are happy with?

Since like forever. If we can concede that grading influences the people being graded then the consequences of grading should be taken into account.

[Aimless]

So here's a philosophical question. Are college studies the same as "most professional jobs"?

[Khendraja'aro]

Since like forever. If we can concede that grading influences the people being graded then the consequences of grading should be taken into account.

Uh, no. Grading is supposed to be a measurement of performance. Doing stuff like that would make them even worse - if that's even possible.

[GGT]

Using SAT scores as a proxy for the preparedness of students, the researchers were able to rule out patterns in which Republican or Democratic professors had better students.

Who makes the SAT tests and decides how those are 'graded'?

Besides, don't all those professors make up their own tests and use different grading systems? Sounds like comparing apples and oranges.

[Lewkowski]

Since when is grading about what people are happy with?

Because grades are not the analytical instrument you make them out to be?

Listen, you can measure the velocity of a car. And even there people are going: "Oh, the instrument was not calibrated!", "The office got it wrong!" and so on and so forth. And that's a simple measurement.

And then we have grades where literally(!) everything under the moon comes into play. And you think that is reliable? Heh. You're even more naive than I thought. And you're the same person who's doubtful of climatology because "it's all numbers and the wrong ones to boot"?

Everything under the moon? Sure things are going to be subjective at times but there is a ton of objective data that can be factored in to a grade. Examples: Score's on a test. Sure if it an essay / short answer test it will be subjective but one would hope the professor would be consistent with how he grades them. Multiple choice of course is easier. Again this is similar to a job situation, of course there will be some subjective things your boss looks at. But hopefully there are some numbers that can be produced as well.

[Aimless]

Uh, no. Grading is supposed to be a measurement of performance. Doing stuff like that would make them even worse - if that's even possible.

For the record, one important reason why many med-schools have switched to a pass/fail system is that they've seen negligible negative effects on performance that are offset by strong positive effects on how students feel. I can see the merits of pass/fail systems. But I'm not going to pretend that such a system is any more flawless than other systems, and for that reason those who're affected should have a say in things. If it makes things any clearer, I'm thinking of this as being somewhat similar to deciding what treatment to choose for a sick patient. There are often several different options, each with their pros and cons. I can't always just unilaterally make decisions

Look, we're talking about adults here. Education is and should be an alliance at that level if it wasn't so before. If nothing else then at least because students are important for teachers' salaries no matter how much teachers hate it

[Aimless]

I'm a little surprised that Lewk doesn't approve of pass/fail systems. I thought a binary either/or black/white system that reduces ambiguity and the chance of error would be right up his alley.

[Nessus]

I'm a little surprised that Lewk doesn't approve of pass/fail systems. I thought a binary either/or black/white system that reduces ambiguity and the chance of error would be right up his alley.

His social darwinism requires constantly watching everyone's performance to gauge their worth.

[Khendraja'aro]

Everything under the moon? Sure things are going to be subjective at times but there is a ton of objective data that can be factored in to a grade. Examples: Score's on a test. Sure if it an essay / short answer test it will be subjective but one would hope the professor would be consistent with how he grades them. Multiple choice of course is easier. Again this is similar to a job situation, of course there will be some subjective things your boss looks at. But hopefully there are some numbers that can be produced as well.

Heh. You know what objective data looks like? It's measurements cleansed of any subjective influence. That's why, for example, when I measure something using an analog display device I always include error margins which not only include the device's error but also the human error.

With grades you can't do that. Yes, you can do nation-wide multiple choice tests with the same questions and answers for everyone.

That's insufficient for many tasks, however. For instance, in Physics the result is as important as the way you got to that result. In languages, you need to assess the way someone uses the language. In Mathematics, you have to prove some relationship. In Chemistry, you're needed to demonstrate exactly how a certain reaction proceeds. And so on and so forth.

Which means, someone will have to assess if your answer is fitting to the question. And that's where subjective enters the playing field.

I read about some experiments about grading. For instance, the background of a pupil plays a massive role in grading - does he look well-dressed? Does he take care of himself? Do his parents possess an academical background? Or is he a slob? Never talks much? His parents working class?
Either of those factors can skew the grade by a whole increment.

Then there's the way a teacher/professor does the grading: Will he use the social norm - which could mean, as an example, a Gaussian distribution of grades. Individual norm - where a student's grades are dependant on his own progression alone? Or a criterial norm - where ones efforts are graded against criterias set-up before the test?

And what does the teacher/professor intend to do with the grades? Are they supposed to be a source for motivation? A punishment? Do they show what the student did the last semester? Do they propose to show what the student could do?

And the list goes on. There are countless influences on grades - and everyone has a different idea of what should influence them. There's no clear definition "Grade A means:...", only a "Grade A could mean that... It could also mean that... And it could also mean a whole different thing altogether!"

For the record, one important reason why many med-schools have switched to a pass/fail system is that they've seen negligible negative effects on performance that are offset by strong positive effects on how students feel. I can see the merits of pass/fail systems. But I'm not going to pretend that such a system is any more flawless than other systems, and for that reason those who're affected should have a say in things. If it makes things any clearer, I'm thinking of this as being somewhat similar to deciding what treatment to choose for a sick patient. There are often several different options, each with their pros and cons. I can't always just unilaterally make decisions

Look, we're talking about adults here. Education is and should be an alliance at that level if it wasn't so before. If nothing else then at least because students are important for teachers' salaries no matter how much teachers hate it

Pass/fail would be better for the reason that it would get rid of this stupid notion that grades are a reliable measurement of any kind.

I mean, look at any study. You see the kind of statistical shenanigans they have to go through in order to show some correlation or other? Where they talk about influences, what data they had to throw out and which they simply couldn't explain?

That would be the requirement for grades to be at least halfway reliable. But we don't do that because it's too much of an effort. And yet we do cling to this notion that grades are somehow scientifically grounded. Because they're grades, you see?

[LittleFuzzy]

I'm a little surprised that Lewk doesn't approve of pass/fail systems. I thought a binary either/or black/white system that reduces ambiguity and the chance of error would be right up his alley.

It's different from what he's familiar with. That means it's wrong. Classic conservative caricature.

[Lewkowski]

Khendraja'aro - So if I can sum up your argument in one sentence what would I say? Its hard to evaluate people therefore we shouldn't do it. Insane. Pass/fail eliminates comparisons with the group. Yes ranking people and rating people telling one group they are better in x y z then another group IS important.

Fuzzy - You're being dumb. I support free markets and for all of my adult life I've never existed in a country that has had true free markets. So that little theory fails.

Nessus - And what is wrong with gauging people's worth? That's rather important in society, business and yes academia.