Validity of Bell Curve distribution of grades

[ckafrica]

Hey All,

The moronic overlords at the private business school I work for are trying to force us to institute a bell curve distribution of letter grades meaning for example that 10% of the class gets an A, 20% get a B, etc. The result this is creating is that we've got classes taking a nearly identical test from semester to semester whose letter scores are drastically lower when numeric scores are higher to simply because they are in a class of bright students (we have a class that routinely scores 15-20% higher than other groups because they are all just awesome)

I'm looking for some ammunition, both statistical and psychological as to why what they're doing and especially how they are doing it is a dumb idea (on simple economics, the students will revolt and withdraw and tell their friends what a bunch of ass-clowns our school is, cutting into our bottom line). Explanations good, academic citations better.

Thanks for the help denners.

Edit: Just clarify the situation; we are a small school (300 students, class sizes of 10-30) in Vietnam where the tuition is 8 times GDP per capita. We get the upper middle class' very bright (know enough not to want to go to a government college) and the ones who couldn't pass the state entrance exams (because money is our only criteria of acceptance) so our students tend to veer towards the extremes rather than the middle.

[angelfromanotherpin]

http://theapple.monster.com/benefits/ar ... urve-hurts

[Count Arioch the 28th]

Only the top 10% of the class is going to matter anyway, I don't see the big deal.

[ckafrica]

Thanks Angel, though I had found that one already.

@Count: The problem is I've got one class who score range is 87-98 when last term it was 45-98. Currently we've got a set numeric range to letter score conversion (which is a tad generous imho but that is another kettle of fish). But with the bell curve distribution, both 98s still get the A+, but the 87 in the smart group gets a D while if they had been the more average group the semester before, it would have been an A-. I think you see why they might get fucked off.

And the teaching staff genuinely care about the students and don't want to see them fucked over just so the wankers in Singapore can have their pretty fucking curve they've suddenly decided they need. Thankfully we've wrestled away enough power over here that we don't just have to bend over and take it but we have to show them why they need to go fuck themselves.

[Juton]

Only the top 10% of the class is going to matter anyway, I don't see the big deal.

I think saying the top 10% of students is going to matter is more accurate, but still not that accurate. If firms are only interested in the top 10% of students, then if this years batch is awesome and last years sucks then the students from this year should expect a 20% employment rate in their field.

My school never decreases a grade when marking on a curve, only increases it. Our thinking is that grades are something a student earns, and it is wrong to take away something someone has earned. Also mandating the percentages of A's has always been described as a recipe for students to use every form of academic misbehaviour to get good grades.

[tzor]

I know my college did grading on the curve for Freshman physics back in the 80's but it was for an entirely different reason. Basically we had one professor for all the classes (combined lecture hall sessions) and teacher's assistants at the lower levels who gave out and administered the tests. The results for all the TA's were tabulated and adjusted with the assumption that the distribution of students assigned to each TA was more or less the same and so any variation of grades was due to the differences in the scoring criteria of the TA.

[K]

My law school did this.

So, in law school you take one test a semester and it counts for 100% of your grade, and was usually worth between 100-200 points.

I had friends who were in classes where everyone got between 86-88 on the test, and the guys who got an 88 got an A in the class and the ones with a 86 got a C.

The difference was literally two points on a 100 point test.

Personally, I think schools that work this way are basically trying to make failures. The fields they train for only hire the top 5% anyway, so all the others along the way are just the suckers who spent 100-150K on tuition and can't get hired.

[Doom]

Hey All,

The moronic overlords at the private business school I work for are trying to force us to institute a bell curve distribution of letter grades meaning for example that 10% of the class gets an A, 20% get a B, etc.
Edit: Just clarify the situation; we are a small school (300 students, class sizes of 10-30) in Vietnam where the tuition is 8 times GDP per capita. We get the upper middle class' very bright (know enough not to want to go to a government college) and the ones who couldn't pass the state entrance exams (because money is our only criteria of acceptance) so our students tend to veer towards the extremes rather than the middle.

I had to sit on an appeals commitee when a fool Education professor actually made grades on the bell curve, for a class of like 15 students. To get an A, you had to score 2 standard deviations above the mean (two below, for an F). Needless to say, his Education students didn't understand a bell curve anymore than the professor did, and there were many complaints.

A good student in the class kept getting higher and higher grades, but, of course, the higher her grade, the higher she needed to score to get an A, she was mathematically not going to get an A in the course...one other student was scoring better, and in a class that size, there was no way to get 2 A's.

While curves can make sense, here are some reasons you don't want to use a bell curve system (it should be noted, you're not actually using a bell curve, you're using a forced distribution that resembles a bell curve, but much of commentary still applies):

1) Courses are not competitions, there is no 'winner'. A grade represents mastery of the material. One person's grade cannot possibly represent another person's level of understanding, but if you use a bell curve for grading, then each student's grade influences the other student's grades.

2) By using a mean/standard deviation distribution, a student failing the course (by, say, getting a 0) can actually prevent another student from getting an A. A very low score will increase the standard deviation even as it lowers the mean, and can easily make it so large that being a certain number of standard deviations above the mean is impossible (i.e., requires a score above 100). (Note, this is in an actual bell curve, a forced distribution won't much have this.)

3) Students need clear goals. A student is told a 90 is an "A", before the test is taken. A student that scores well will raise the mean, to, say, 91. The act of getting what was supposed to be an A can actually prevent the student from getting an A. Similarly, if enough students get really low grades that should be failing, they'll end up all getting C's.

4) You need to be able to compare students in different classes. Imagine a class where most everyone gets a 60, but one student, top of the class, gets a 70. He's an A student in a bell curve system. In another class, most folks score around 75, and the top student scores s 95. He gets an A, too. Do the top students in both classes really have the same level of mastery?

5) In small classes (under 20), it becomes very difficult to give an A, or fail a student, depending on how many standard deviations away from the mean you use for top/bottom grades. Once you drop below 10, it gets downright silly in practice.

There are other reasons, but that should be a good enough start.

[Ancient History]

Engineering students do not do bell curves. That shit would result in many nasty, college career-ending lab accidents.

[Gx1080]

Besides all the reasons that Doom exposed, Bell Curve distribuition of grades massively encourages students to try to screw each other for said grades.

Unless the whole point is to graduate sociopaths, I mean, people adapted to the corporate workplace, which wouldn't surprise me. Then is A-OK.

PS: Count, your prole resentment is douchey.

[Count Arioch the 28th]

Your MOM is douchey.

[K]

Engineering students do not do bell curves. That shit would result in many nasty, college career-ending lab accidents.

Law students are famous for sabotage.

For example, razorblading critical pages out of the research materials in the law library was a favorite, but you'd be surprised how often laptops get stolen or destroyed in the law library (which has security to keep non-law students out, so it had to be law students doing it).

I find it slightly disturbing how many potential lawyers are willing to commit crimes to beat the curve.

[Whatever]

There's more: moving books to other shelves is a nice trick. No permanent damage, but only you can access it now (unless someone wants to search the whole library for a mis-shelved book).

Having two sets of notes, one to share, and a real set for personal use, is another trick, but it can be rather obvious what you're up to.

This is mostly an issue at the lower ranked schools, where you have to graduate at the top to get any kind of job. Conversely, you can graduate Harvard Law at the bottom of your class, and you'll still do fine in the job market if you've made decent connections (and you have, it's Harvard), so there's less pressure to cheat.

[RobbyPants]

I find it slightly disturbing how many potential lawyers are willing to commit crimes to beat the curve.

It is disturbing, but I guess it's totally expected if it's encouraged that strongly.

Chalk this up with many other systems that had good intentions and unintended side-effects. Doubly so since the side-effects are likely worse than the benefits of the system!

[cthulhu]

Given that the difference between a top law job and a middle of the road one is about 80-100k P.A. yeah, that's a fuck ton of money.

You're essentially fighting for an instant cash payment of $150k.

People are going to fight for that.

[Neeeek]

Engineering students do not do bell curves. That shit would result in many nasty, college career-ending lab accidents.

Law students are famous for sabotage.

For example, razorblading critical pages out of the research materials in the law library was a favorite, but you'd be surprised how often laptops get stolen or destroyed in the law library (which has security to keep non-law students out, so it had to be law students doing it).

I find it slightly disturbing how many potential lawyers are willing to commit crimes to beat the curve.

I've heard of this kind of thing, but it doesn't jive at all with my experience in law school. Where I went, you could leave your laptop completely unattended for hours and no one would touch it.

Honestly, I've never been around a nicer, more honest group of people than my fellow students in law school.

[Count Arioch the 28th]

The people that sabotaged my schoolwork the most were in middle school, and the crappy no-name community college I went to. I would think that a gen-ed brit lit course wouldn't be on the list of places where your classmates would try to fuck your shit up, but there you go.

I am not smart enough to be anywhere near a law school, so I can't speak if that was worse or not.

[PhoneLobster]

In NSW Australia what amounts to the final senior high school college qualifying exams (and all related assessments for two years) are graded using a (huge and insanely elaborate) multi faceted bell curve system so complex as to be just short of completely opaque to even the most determined of experts.

It has been this way for a couple of decades punctuated ONLY by small arcane and poorly understood (and totally un-advertised) changes.

The system is SEVERELY fucked up and I have heard many of the following claims, from teachers, trying to advise confused and angered students, any or all of which may be true, false, or true or false depending on year by year changes to methodologies.

1) Your grades in the final exams would go down if your teacher gave your class good grades on their prior assesments.
2) Your final result would go down if your class did poorly. (we had drunk guys turn up, write their names on the paper, pass out for five minutes then leave!)
3) Your final result would go up if your class did well.
4) Your final result would go down if your SCHOOL had a history of lower marks.
5) your final result would go UP if your SCHOOL had a history of higher marks.
6) Your final result would go DOWN if your state wide YEAR did too well.
7) Your final result would go UP if your state wide YEAR did poorly.
8) You would get a higher mark for being in a Private School (they use insider knowledge to basically help their students game the highly dubious system).
9) You can sometimes get a higher qualifying mark to get into your college level course (like law or medicine or whatever) by doing crappy low level retard versions of science and math courses instead of proper biology or whatever.
10) You can sometimes increase your final grade by doing a level of course well over your actual capabilities, as long as everyone else craps out on the marks as well.

In the end the surest way to get a good mark was to luck out and do the right combination of courses such that everyone else did poorly that year and you did well, then your results would suddenly catapult you well beyond all the SUCKERS who did even better, but were in courses that did well accross the board that year. Unfortunately there was no way to know in advance whether you should do "Math for Retards", "Mega-Advanced-Really-Hard-Version-Biology", "Political-Assing-About-Free-Time-Class" or "Stupidly Hard Drama".

Any one of those choices could, year by year, guarantee a competent student a place in the state wide top 100... or complete failure to enter a college level course they are thoroughly qualified for.

The HSC is awesome.

[Maxus]

PL, I dig that Australia is very tolerant, even proud, of its alcohol consumption.

But seriously. People need to sober up for -some- things. That could only have been conceived by a ton of drunk people who remained stinking drunk for an extended period of time while all of them were working on the same project. In the same room.

That's like, Drunk Art By Committee when said art is a new building.

[PhoneLobster]

But seriously. People need to sober up for -some- things. That could only have been conceived by a ton of drunk people who remained stinking drunk for an extended period of time while all of them were working on the same project. In the same room.

As I understand it that is not how they design the HSC.

That's how they do the raw marking process for the final exams.

THEN that raw mark is taken and it runs through state, school, subject, class, assignment history, funding history and various other elaborate bell curves and other opaque weightings.

I am convinced it isn't a means of academic grading but instead a fortune telling mechanism. If someone told me it included a bell curve weighting by star sign I would not be entirely surprised.

Edit: Back when I did the HSC the biggest impact on my marks was that a number of my subjects had final exams that were a teeny bit too easy that year. Too many people graded too high, we all copped a pretty big "correction" as a result. It is absolute insanity but the result was I was sitting there doing my Chemistry test and thinking "Oh crap, these questions are way too easy my final net grade (for chemistry and thus for the over all final combined mark too) is going to go down because I am correctly answering these questions without enough difficulty.

[cthulhu]

Eh, HSC isn't that bad. The process is linear but complex, with many steps. Most of the complexity arises from the fact that the system plots the relative scores of every school leaver in the country against each other - even if you studied English/Religion/Maths for Retards/Some other random shit and I studied English/Advanced Maths/Physics/Chemistry.

The biggest gotcha is that the results the rest of your class gets realtive to yourself both in the subject AND on standardised testing is used to caculate your final score, which gives private schools a huge edge because they can much more aggressively stream pupils to keep poor students out of tough subjects. In private schools non performers are typically asked to quit the toughest maths and science subjects in year 11 if they are projected to score poorly in the standardised testing in year 12.

The problem is that many retards are involved (and in authority positions! I've had teachers flat out explain it to me incorrectly and who didn't understand enough maths to understand the correct explaination) who don't understand how it works and spread fairly deliberate misinformation about it.

It is complex though, and this complexity combined with the plague of retards that seem to infest public life you get the complete insanity. Further more, it's like the Australian electoral system. While it is theortically possible to return someone other than the conchordat winner with tactical voting, actually making this happen is difficult. With the HSC trying hard as you can in the subjects you want to do is usually good enough (with the caveat that soemtimes selecting a harder subject is better because of the streaming factors in the gotcha above).

If you really want to know, the process is as follows:

1) Firstly, your raw HSC mark used for scaling is calculated from your Raw Examination Mark and your raw moderated school assessment mark.

The HSC mark to be scaled is the average of your Raw Examination Mark (i.e. the mark you received in the external exam) and your raw moderated school assessment mark. The latter is determined through the process known as the HSC moderation process. (We recommend that you read more on moderation if you don't know the process of how school assessment marks are treated)

This raw HSC mark will be scaled according to that particular subject's scaling for that year. The scaled mark arrived at will be out of 50 per unit of study.

Relationship between UAI and Aggregate

2) UAC takes the best 10 units (i.e. highest scaled marks for 10 units) including at least 2 units of English and adds up all scaled marks to form an Aggregate mark out of 500.

For example, if after scaling, your scaled marks were: 92/100 for English Advanced, 93/100 for Chemistry, 93/100 for Physics, 99/100 for Maths Extension 1, 99/100 for Maths Extension 2, your aggregate would be 92 + 93 + 93 + 99 + 99 = 476 / 500, (those marks correspond to top 1% in all subjects) and your UAI would be around 99.95. Note that if you do English Extension 1 or 2, any combination of 2 or more units of English can count to your UAI. For example, 1 unit from English Advanced, 1 unit from English Extension 2.

3) The Aggregates of the entire cohort are compared and UAI's are assigned corresponding to the percentile of each Aggregate.

The percentile position of each Aggregate score relative to the entire cohort (including those who have left school at the end of year 10) corresponds to the UAI, after rounding to the nearest 0.05 intervals. The final adjustment process effectively accounts for how the early school leavers would have done had they continued schooling and received a UAI. Typically each year there are 21-23 students who achieve 100 UAI, followed by around 41-43 students for each 0.05 UAI increment thereafter.

So in simple terms, if your aggregate places you in the 99.85th percentile among your entire cohort, your UAI would be 99.85.

More detail than you could ever possibily want here: http://www.duxcollege.com.au/hsc-scalin ... sc-summary

[PhoneLobster]

I went through under the TER incarnation.

It was, I am led to believe, somehow different to the modern "reformed" scheme (which is still pretty batshit crazy). But in ways that no teacher within our school felt compelled or for that matter capable of explaining and that no teacher I know (and I know a rather excessive number of teachers for family reasons) is capable of explaining now.

It's worth pointing out the worst bad guys of the quoted system...

The latter is determined through the process known as the HSC moderation process. (We recommend that you read more on moderation if you don't know the process of how school assessment marks are treated)

Which is arcane speak for "we will fuck you over if you go to the wrong school".

With a hint of extra "We will fuck you over if we get a bad feeling about the internal assessment marks your teacher hands us".

I'm led to believe the internal assessment mark bit is at least different from when I did it, but not necessarily new.

This raw HSC mark will be scaled according to that particular subject's scaling for that year. The scaled mark arrived at will be out of 50 per unit of study.

Which is secret code for "we will fuck you over if you get the yearly subject pick lucky dip wrong".

The Aggregates of the entire cohort are compared and UAI's are assigned corresponding to the percentile of each Aggregate.

"And then we will really really screw you over if your year as a whole does better than prior years."

The final adjustment process effectively accounts for how the early school leavers would have done had they continued schooling and received a UAI.

Which I THINK is new since I did it. And also has really nasty interactions with poor area schools, our harsh unemployment system and student welfare programs. So again its a reminder "We will fuck you over if you go to the wrong school".

You will note that official summary actual skims over and does not explain the mechanisms behind some of it's nastiest parts. Like assessment moderation and subject scaling.

[cthulhu]

I do agree with PL that the school you go to can fuck you in the ass with the HSC system (mostly because it becomes essential to score two standard deviations above your class if they are retarded in testing which is bloody hard work).

This is the feature I dislike most about it.

I went through under the TER incarnation.

It's basically the same , the only thing that has really changed (in NSW) is the balance between in school assessment and the final exam.

But in ways that no teacher within our school felt compelled or for that matter capable of explaining and that no teacher I know

Yes, the only good summaries tend to come from good mathematics teachers because of the amount of statistics involved in the end result.

Which is arcane speak for "we will fuck you over if you go to the wrong school".

No - this is an important balance check. If the results were skewed the scaling factor would undo any effect here. this step is created to try and STOP massive scaling between schools by setting the same standards across all the schools for the same subject. It does not, of course, work properly hence the scaling between schools.

However, it cannot fuck you because if your class can pull it out it out during the standardised test you will get marked back up.

Which is secret code for "we will fuck you over if you get the yearly subject pick lucky dip wrong".

They publish the results, and the subject pick lucky dip rarely (if ever) changes. The top 5 subjects for most advantegous scaling I don't have think have changed ordering in the last 10 years. The table is published every year by the NSW government. Everyone knows that the hard subjects scale better too (as well they should).

"And then we will really really screw you over if your year as a whole does better than prior years."

It doesn't rarely matter though because the only purpose it exists for is university entry and the overwhelming majority of students will apply to enter university the same year. Any effect from this (and I doubt there is one.. but whatever), is more than drowned out by the massive shift in entry scores from the universities themselves.

Which I THINK is new since I did it. And also has really nasty interactions with poor area schools, our harsh unemployment system and student welfare programs. So again its a reminder "We will fuck you over if you go to the wrong school".

No it doesn't. It really has no material impact AND is assessed on a statewide basis - the school you personally go to has no effect here.

You will note that official summary actual skims over and does not explain the mechanisms behind some of it's nastiest parts. Like assessment moderation and subject scaling.

It's a school's own summary (hence the URL). The offical summary is here: http://www.boardofstudies.nsw.edu.au/yourhsc/ and goes into considerably more detail over many more pages.

[PhoneLobster]

No - this is an important balance check.

Back when I did this the way it worked was that our teachers were given a ration of marks according to bell curves and told they couldn't give out more or less than that ration regardless of assessment results.

These days I am told teachers are expected to predict the sort of spread of marks their class will get in the final exams and give similar spreads of marks in the (prior) assessments. And if those two sets of marks don't match up nicely, bad things happen.

They publish the results, and the subject pick lucky dip rarely (if ever) changes.

That seems odd. Because we were never informed of subject scaling in advance back when I did it (prior to 10 years ago). Indeed the closest we got was to be told that the then 4 unit math subject was "usually scaled very high" but that subject scaling was not known until the results of the cohort in that subject came in. Which meant that scaling was basically not even calculated until after the final tests. And I know for a fact that, and several other key "advanced" subjects that year had much worse subject scalings than they had in other years.

Certainly to this day I speak to actual students and teachers and they do not know what the subject scaling is so even if they changed that since I did it clearly something STILL isn't working here.

It doesn't rarely matter though

It matters a lot. I got out fairly OK because I was in the top percentile of my (crappy) school, but it makes or breaks things for many of these kids. Universities DO apply their own school based and regional scaling in an ad hoc attempt to partially compensate for the bad scaling effects of the HSC (I mean surely THAT is a sign the scaling system is bad if it needs RESCALING before being applied).

And no there ARE comparisons made across years. Failing anything else the kids themselves get to feel like shit when they pump in one of the biggest efforts of their lives into the fields and subjects they have the most talent and interest for and get a number in very large part outside of their control and not directly connected with their actual talents and achievements as a result.

And again, it's not like it's even all that helpful for the Universities that then have to jumble all the numbers up again and ignore portions of them for the whole "full fee paying student" debacle and so forth.

I can't see this sort of system being retained a generation from now, for obvious reasons.

[K]

I can't see this sort of system being retained a generation from now, for obvious reasons.

Right now, it's hard to for me to see the current educational system as little more than a mass disenfranchisement system.

I mean, the number of people I know with advanced degrees who are unemployable in their own field is quite large. The amount of debt that young people are saddled with is huge (often enough to buy a house or start a business) and the value they get seems to be statistically biased (ie. the people who got degrees were going to be successful in whatever they did due to intelligence and work ethic and the degree they got didn't account for their higher lifetime earnings).

For example, it's simply become a matter of course for law school grads to not be employable as lawyers with only the top 5% getting jobs (a little better in Ivy League schools, but not by much). My first lawschool roommate killed himself after graduating because of his bleak prospects and the rest of my friends are either completely unemployed or being supported by their parents while pretending to be solo practitioners.