Sunday, August 23, 2009

What we've got here is a failure to evaluate


This morning, I visited the SOCCCD “almanac,” and I checked out some of the curious info available there.

In particular, I came across a page that presents the distribution of grades in the various divisions and schools of our two colleges. (See Grade Distribution Matrix.)

For some reason, the matrix concerns only Spring ‘06, i.e., three years ago. But that's OK. I figure that, since the population of instructors doesn't change much from year to year (yes, we've done lots of hiring recently, but do the math), it is likely that these distributions are fairly constant. And so I would bet that the '09 distributions are close to these '06 distributions.

The most commonly assigned grade: A

Well, call me Mr. Oblivion, but I was amazed by what I found. “A” is by far the most common grade awarded, followed by “B,” then “C.”

Shouldn't the grade distribution resemble a bell curve? I.e., shouldn't A's be less common than B's, which in turn are less common than C's? Shouldn't C be the most applicable grade for the hoi polloi?


In the matrix, “F” grades are twice as common as “D” grades. That makes sense to me: my guess is that this reflects the large number of students who, upon doing poorly, simply bail, thereby ensuring an F even though, had they remained, they would have had a shot at a D or C.

E.g., Liberal Arts and Humanities:

Let’s look at some of the details, focusing--for no particular reason--on Saddleback's division of Liberal Arts and IVC's comparable School of Humanities and Languages. Why the hell not?

Saddleback College:
Liberal Arts:

6,981 grades given.
Grade distribution in percentages:
A: 34
B: 24 (A&B: 58%)
C: 12
D: 4
F: 7 (D&F: 11%)

Incomplete: 1
Credit: 14
No-credit: 5

Irvine Valley College:
Humanities and Languages:

5,027 grades given.
Grade distribution in percentages:
A: 31
B: 21 (A&B: 52%)
C: 12
D: 4
F: 9 (D&F: 13%)

Incomplete: 0
Credit: 13
No-credit: 9

Gosh, Saddleback instructors sure do hand out lots of A’s! But IVC faculty manage a pretty close 2nd in the "A" giveaway sweepstakes.

"Ah," you say, "nobody's surprised when humanities instructors give away A's. What about the other areas of the colleges?"

Here, then, is the data per college:

Saddleback College:
A: 37
B: 23 (A&B: 60%)
C: 15
D: 5
F: 11 (D&F: 16%)

I: 0
C: 7
NC: 2

Irvine Valley College:
A: 35
B: 19 (A&B: 54%)
C: 13
D: 5
F: 12 (D&F: 17%)

I: 0
C: 10
NC: 6

Click on image to enlarge

When we look at the colleges as a whole, the "A" giveaway phenomenon gets worse, not better.

What we've got here is a failure to evaluate, or something like that. No way do these grades reflect the competence of actual students. Yes, yes, I know: some students do very well. They deserve their A's no matter how you cut it. But the vast majority of our students are, well, distinctly non-excellent. In truth, there should be lots of F's, D's, and C's. A's and B's should be relatively rare.

But, in fact, A's are by far the most common grade. (Note: at many private colleges, virtually all grades are A's and B's. Gradewise, compared to those institutions, community colleges look positively miserly!)

The pattern we are seeing here goes beyond "grade inflation," for instructors issue grades as though the vast "middle" of studentry were excellent and only outliers can be found on the trail down to B, then C, then D.

Preposterous!

Instructors enjoy playing "Santa Claus"?:

Tell me if I'm wrong: What we are seeing here is (a) a disinclination on the part of faculty to designate students' work "mediocre" or "average"--even when it is--and/or (b) an inclination on the part of faculty to designate students' work "excellent"--even when it isn't.

Call it the Santa factor--or the "Santa, and if not Santa, then at least not Mr. Meanie" factor.

It's everywhere!

Naturally, there are differences between areas: compare, say, the life sciences and, say, the social sciences. But these differences are not as dramatic as one might expect. Further, this perverse pattern or something nearly as odd afflicts all academic areas at our colleges.

Obviously, one would want to compare these distributions with those at other colleges. (My impression is that the teacher-as-Santa phenomenon is very common, but it is more pronounced at private colleges. For instance, at Brown University last year, a majority of undergraduate grades were A's,)

I wouldn't be surprised if we were to find pretty much a duplication of the above at other local districts. (If I find the time, I will look up the data.)

Whence giftage?

The reasons why instructors grade as they do are various and complex. It's not just the desire to play Santa (or to not play Satan). As we've noted previously, research shows that today's students expect to receive A's and B's just for showing up.

Really.

Hence, instructors who call a spade a spade pay an unceasing price in student anger and disgruntlement. That can be taxing.

And given the context of routine A giveaways, any instructor who gives A's and B's only to good students and assigns D's and F's to poor students will likely become unpopular.

Driving away students: that's no small thing. At the very least, it's bound to be demoralizing.

And then there's administrative "leadership":

Meanwhile, the community college system tacitly (explicitly?) promotes the idea that students can be fully employed, carry on a significant social life, and take a full load of courses all at the same time. After a while, students assume that doing little-to-no homework and attending half the scheduled classes is normal. Alas, that seems to be what many students are indeed assuming.

I've tried insisting that students do 6 hours of homework per class per week. Forgetaboutit.

And so our system daily promotes student decline and delusions of competence.

See also
Grade Inflation Seen Rising (Inside Higher Ed, 3/12/09)

This article reports the results of a study done by Stuart Rojstaczer, a retired Duke University professor:

Rojstaczer's findings will probably resonate with professors, many of whom regularly bemoan grade inflation and say that students are conditioned to expect good grades just for showing up, and that professors who refuse to go along get punished with harsh course evaluations. Many professors who are off the tenure track or who are pre-tenure report great fear of being punished by students (and then not rehired) if they gain a reputation for tough grading, and studies have found correlations between being an easy grader and earning good ratings at RateMyProfessors.com.…

Grade inflation—and deflation—in higher ed (DtB)

Students fail--and professor loses job (Inside Higher Ed)

Wikipedia on "grade inflation"

Wikipedia on "grades"

34 comments:

Anonymous said...

Is there any difference in the math and science divisions, where the grading is probably much more objective?

Roy Bauer said...

The data for Math, Science, and Engineering (at Saddleback):

A: 22
B: 24
C: 25
D: 10
F: 16

I.e., there's no curve for A, B, C. They are equally numerous. This isn't good, but it is better than the perverse pattern in many other areas.

Roy Bauer said...

I just checked, and the '06 data for IVC's Math, Computer Science, and Engineering returns us to perversity: A's are much more numerous than B's. (On the other hand, generally speaking, the perversity phenomenon is more pronounced at Saddleback than at IVC.) But you really should peruse the data yourself. Just click on the link provided.

And remember: we're looking at '06 data.

Roy Bauer said...

P.S., check out the data for PE at Saddleback. Whoa! In comparison, IVC's crew look draconian.

Anonymous said...

What is the history behind student evaluations? When and how did they become an influence on grading practices? I think it must not have always been like this, but I don't know. Roy, how has it changed (or not) over the years you have been teaching? And if it has, when why and how do think that has come about?
ES

mad as hell said...

Grrrr.....The phenomenon of faculty giving away A's like that just makes my blood boil. I have, indeed, faced shock, tears, reproach, anger, and nasty evaluations myself, from those who expected A's (doubly foolish in Philosophy!) and instead received C's that were almost gifts. That's incredibly annoying and disheartening, even when one has tenure. (I'm not just speculating that the nasty comments were about grading; the students say so in so many words, often.)

Thanks for highlighting this problem, Chunk.

Anonymous said...

Egads.

Anonymous said...

ES, I am not familiar with the history of student evaluations of instructors, though, from my somewhat parochial perspective, student evaluations have always been around (my higher ed experiece stretches back to 1973). During that time, generally speaking, academics have wisely resisted the notion of relying entirely or primarily on student surveys. On the other hand, the notion that student surveys should be one element in faculty evaluation seems here to stay. I have come to believe that student surveys are very valuable if read carefully. (One major problem with Rate-Your-Prof, etc. is that the samples are likely skewed toward extremes of opinion.) I have learned much from student surveys, especially in the early years of my teaching career. (For instance, I no longer beat students with sticks or have students take turns screaming their primal fears and desires.) But of course students typically do not have the discernment or sophistication to assess all that might be evaluated. In recent decades, there has been advocacy of "peer" evaluation, and that seems to work very well at some institutions. (In my experience, academia is a zone of high integrity. Cheating and scamming and fudging are at worst rare.) SPECULATION: My guess is that student surveys (etc.) emerged along with many other idealistic and (seemingly) progressive notions during that largely embarrassing era known as "the 60s." For instance, I know of one small private liberal arts college that started a special program during that time that eschewed grades and embraced various flakoid activities and ideals. I think it may still exist, though, by now, that faint stink of marijuana and incense is redolent of stupidity and excess for virtually everyone. -R

Bohrstein said...

I like reading about this problem. My personal experience comes mostly from the physical sciences area where math professors at IVC are "hardcore," though some DO speak of inflation. I have small tale pertaining to inflation:

There is a professor in physical sciences whose lowest grade is a C. This meant, in my classes' case, that 50% is a C. Also you get 50% on your quizzes just for putting your name on the test (in math I get 1 point). I once received a pity A on a midterm. I scolded the professor a small amount, but he assured me I was deserving of it.

I lived Physics for the rest of the semester because of that, I was so ashamed. That pity grade insured that all I was going to be doing was getting fair grades. As nutty as it sounds, I recently cleaned out my room and threw away all of my old tests. I kept the bogus one, and retook it. Easy as hell, I'll take my A. I had messed up on one problem, and it was a conceptual issue, one that I can't believe I made. I figure, perhaps incorrectly, that the important thing is that I know the material. Of this material on this test, I can safely say I do.

running late for work, but wishes all a freakin' awesome first day BS

Anonymous said...

Well, that explains why I was somewhat disappointed with your courses, Roy. I realize now that I was very much looking forward to the sticks and primal screaming. Had I known that you no longer favored these valuable and proven methods of teaching, I likely would have enrolled in some other class. I can only imagine that Bohrstein shares my feelings on this.
ES

mad as hell said...

At my U, we have an excellent review process that includes reams of student evaluations (every one of which our personnel Committee actually reads), but also peer evaluations from each of one's colleagues, and also peer comments *on* the patterns manifested in the student evaluations. At least in the old days, the Committee and one's peers were very good about sifting through student comments, noticing patterns, and then offering explanations of the patterns that either did or did not suggest that the person's teaching was lacking. For example, if almost all students thought the course was easy (not challenging), that was a red flag for peers. If students were disgruntled about "hard grading," peers took that as a good thing. Between the good judgment of experienced peers and a variety of careful questions on eval forms, this system seems to have worked pretty well.

We've had stupendously popular profs who received negative reviews or were refused a promotion or tenure in spite of student adulation, and we've had unpopular teachers who were promoted (for all the right reasons)--pretty reassuring.

Still, I feel oppressed at the weight given to student evals at my private institution, and by the students' perceptions of that weight. And it's still annoying as hell to elicit anger just by maintaining decent standards. At those times I feel that many of my colleagues have failed me by setting me up for unjustified rage.

Anonymous said...

Indeed, the fellow requested a private screaming session and I was forced to turn him down. In the end, he had to settle for hitting himself in the face with his own stick, alone, yelping inarticulately and to no purpose. That's when he switched to business. -R

Anonymous said...

M.A.H., I sympathize. This regard of student opinion is a kind of political correctness, I think, which is to say it is the product of activating a "no thinking" switch for the sake of some holy bovine. Hate it. Still, it sounds like your institution has managed a set of practices that is in the ballpark of wise and sensible. Tell you what: you should nominate yourself "teacher of the year," generate oodles of fake letters of support, win the prize, then use the cash to fund a lavish trip to Palm Springs, where, I hear, male strippers are college educated.

Anonymous said...

Isn't Palm Springs where our trustees go with their sweeties to relax?

Anonymous said...

If this is so, then it is clear you have failed Bohrstein completely. Despite your good intentions, and I do believe they were good, this unnatural obsession with student evaluations has caused you to alter, perhaps forever, the course of this young man's life! You think that by not beating your students, and that by stifling their primal screaming, you are a better, kinder teacher. But you have lost your way, Sir! Your sacred duty is to impart Knowledge, but by refusing Bohrstein you have instead condemned him to a dark and uncertain existence.
ES

Roy Bauer said...

Pace, good ES, pace. I shall hunt the fellow down and knock him senseless. Tragedy may yet be averted!

Bohrstein said...

Take another class? Hah! Upon entering the mysterious gates of Irvine Valley College, I vowed to destroy the first professor my eyes layed upon. Enter Roy Freakin' Bauer. Whilst I underestimated this swarthy opponent I feel that I have whittled him down over the years. He thinks he is hunting me? Fool! It is HE that is the hunted.

Besides, I make a wonderful business major. All that BS I can spew out, not to mention, my saucy forehead.
- BS

Anonymous said...

Hilarious! And this is only Monday!

MAH

Roy Bauer said...

Swarthy? Freakin' swarthy?
I take umbrage, sir. Yes, umbrage!!

Anonymous said...

Oh, how I was calmed and comforted by your assurances of Bohrstein's imminent drubbing! Alas, I fear you will be too late. Already he is carried away by wild ideas of his "saucy forehead".
ES

Anonymous said...

I'll take your umbrage, which is but a piffle, and will gladly exchange it for my heightened state of miffed vexation.

Good day, sir!

(Sniff with a head turn.)

Anonymous said...

Kind sirs, I beg you to desist, and call a halt to this unseemly verbal duelage. You impede my work with too many a stifled peal of mirth!

MAH

Anonymous said...

I join in your aggrievement, good madam, in that these base louts do, with their unceasing twaddle, unwittingly endeavor to hinder my labors.

Anonymous said...

MAH, even though I've never met you and so have no idea what you look like, I somehow got a visual of your "stifled peals of mirth"! Too funny! Don't stifle too much though - mirth is good for the soul and good for your health... ;)
ES

Bohrstein said...

Dissident foul! You cry like babes in the night!

Anonymous said...

Wretched fellow1 It is not enough that you have inspired in your good Teacher such high dudgeon, he who was once so dear to your own heart, and who has loved you faithfully as a loyal friend! Not content, you now direct your asperities and insults to MAH, who has never done you an injury.
ES

Roy Bauer said...

Gosh, I'm outclassed by this crowd.
I'm throwin' in the towel.

Bohrstein said...

I think a DISSENT fencing club is in order.

Roy Bauer said...

Great. I've got bails of chain-link fence I can bring. I've even got 50 or so pickets, all white. I thought I was the only person who enjoyed fencing.

Bohrstein said...

Holy shit. Was that a pun?

Bravo sir, I don't want to discourage this kind of thing, so I say, "Bravo!"

Anonymous said...

I've always wanted to take up fencing!
ES

Anonymous said...

This seems like the time to work in something about the pen being mightier than the sword, but it's too early to be clever (for me)!

Anonymous said...

ES, I thank you kindly for your spirited defense--but fear not: BS's execrable dismissal above hath pricked me not. Forsooth, truth be known, many an eruption of mirth hath indeed burst forth loudly in the fortuitous privacy of my insulated study. Your concern is most generous, your advice well-taken.

I await impending announcements of said fencing club.

--MAH

Bkwrmi said...

I'm currently a student at Saddleback College, and I have to agree that I don't like the trend. I work hard for my grades, whether it's nessicary or not (and from what y'all are saying, it probably isn't). I get A's, but because of grade inflation, I can only hope that I've actually earned them.

Roy's obituary in LA Times and Register: "we were lucky to have you while we did"

  This ran in the Sunday December 24, 2023 edition of the Los Angeles Times and the Orange County Register : July 14, 1955 - November 20, 2...