Sunday, October 18, 2009

The Bell Curve

Businesses all over the world conduct a ritual known as the Annual Review. The Annual Review takes on many forms and many implied meanings. Some annual reviews are conducted on or about the service anniversary of each employee; others are conducted en masse.

Some reviews are designed to provide feedback on an employee's performance based on a set of goals established the previous year, others are designed to rate employees to establish a basis for compensation.

The King of All Reviews
When I was at Bell Labs, we went through this incredibly rigorous process that took days (not the use of the word incredibly). Each employee would submit a list of their accomplishments for the year. We called it the Why I'm Great! form. Next, taking the Why I'm Great form as input, each manager would complete forms for each employee including accomplishments, strengths, weaknesses, and so on.

We would distribute all the forms to all the managers within a given area and then convene a series of meetings whereby we would rank order (i.e. from 1 to n) every employee in the organization.

As we rank ordered the employees, we would pursue vigorous discussion of the employees strengths, weaknesses and areas of improvement. We would talk about development plans and so on.

Adding a Little Pressure
At times, the discussions would get quite heated. The reasons were three-fold.

First, once we established a ranking for each employee, we would discuss salary treatment. The reason we called it salary treatment rather than raises is that the process was designed to align compensation with performance; ostensibly, the highest ranked employee would have the highest salary and the lowest ranked employee, the lowest salary. Makes sense, right?

The problem was that the rank ordering didn't always reflect the salary ordering. Some employees might have had an off year, others might have transferred from another organization where they were paid more highly or more lowly, and so on.

Since we had a limited pool of funding for raises, truly aligning salary and performance meant providing some people with negative raises. If I had a highly rated person who was poorly paid, in order to fund their salary increase, I would have to take salary away from a poorly rated person who was highly paid. You can imagine the discussions that would arise when a manager was told that one of his or her employees would be receiving a significant negative raise.

Second, in order to be promoted within the organization, one would need to be consistently highly ranked. So, we often had lots of debate over the people who were in almost in or just in the top ten percent.

Third, as a matter of policy, we would look to let go people who were in the bottom ten percent. Again, lots of heated discussion about who deserved to be let go.

All in all, I really enjoyed the process. I liked the debates and what they revealed about both the topic of debate and the debaters. I also really liked the concept of aligning compensation and performance.

The Bell Curve
One of the the most amazing things I discovered in the Bell Labs annual review process was that the distribution of employees' performance levels complied with what's referred to in statistics as normal distribution. When you plot data that distributes in this manner it has the appearance of a bell, so we often call it Bell Curve distribution.

In bell curve distributions, the amount of data decreases as you move away from the center of the bell and increases as you move towards the center of the bell. The farther you move from the center (either to the left or to the right), the less data you'll find.

You might have heard people in science or statistics referring to standard deviations. A standard deviation is simply a way of referring to clumps of data within the bell. If you start at the center of the bell expanding simultaneously to the left and to the right until you've included 68% of the data, you have one standard deviation. If you keep expanding evenly to the left and to the right until you include 95% of the data, you have two standard deviations. At 99.7% of the data, you have three standard deviations.

There's a lot of math that people use to compute this, but in the end, that's all standard deviations mean.

Data that falls into one standard deviation is referred to as being one sigma data, data within two standard deviations as two sigma data, and so on.

So What?
When you think about it, the whole idea of a bell curve distribution seems kind of silly.

If I were to measure the height of every tree in a given forest where the shortest tree was ten feet tall and the tallest tree was 100 fee tall, why wouldn't I find just as many ten foot trees as I did twenty for trees as I did fifty foot trees, and so on? Or, why wouldn't I find all the trees were either ten feet tall or 100 feet tall, with no other heights?

If I were looking at SAT test scores, why wouldn't I expect an equal distribution of poor scores, moderate scores and high scores?

The answer is, "I don't know!"

Nonetheless, as we collect countless data on observable natural phenomena, they seem to conform to bell curve distribution.

It's pretty amazing.

Back to Annual Review
The crazy thing is that Bell Curve distribution seemed to apply to the performance levels of employees at Bell Labs.

Why is this crazy?

It's crazy to me because every one of the people being rated had always been in the three sigma category. They were at the top of their class in high school, at the top their classes as Berkeley and Stanford and Cal Tech and MIT, and so on. They were all used to being the best at what they did. And still, when we did our rank ordering, everything complied with Bell Curve distribution!

The three sigma people (the highest rated and the lowest rated people) were easy to identify. Accordingly, the people in the middle (the one sigma and two sigma people) were also quite easy to identify.

In the end, although we could spend weeks trying to figure out the linear rank ordering of each and every employee, all we really needed to do was decide into which standard deviation they fall. Within each category, the differences weren't that meaningful or significant.

So, How Does this Apply to Me?
If you've hung in here this long, congratulations and thank you!

If you think about it, when it comes to decisions in our daily lives, most of spend most of our time focused on the nuances that distinguish items within a certain group and almost no time on selecting the group itself.

I know managers who will agonize for months or years about a poorly performing employee trying to figure out how to get him or her up to snuff, when in fact, the employee is simply not a match for the job. Alternatively, I have three sigma friends who have miraculously found a compatible three sigma girlfriend or boyfriend, and yet they harp on all minor incompatibilities.

In the end, I think the most useful approach is to understand into what deviation our challenges fall, and then look for a solutions that are in the same deviation. If you have a one sigma challenge, then all you need is a one sigma solution. If you have a three sigma challenge, then don't waste any time looking at one sigma solutions, go right to the three sigma category.

You can apply this approach to everything from thinking about where to eat dinner to what car to buy to where to send your kids to school to whom to marry! By matching the standard deviation of the challenge to the solution, you can make better, faster decisions, and you won't waste time on solutions that will never ever work.

If you made it all the way to here, thanks for your indulgence!

Happy Sunday


  1. I followed you all the way - until I got confused in the sigmas.

    Letting go of the standard deviation and just picking what ever distribution is appropriate with the data, I can see that it is usefull to decide which segment to go for.

    If I want the top job, partner or resturant, I'll pay for it by putting in the effort or the money (I never go for the best seat in the airplane - I go for the best seat in the lower price range).

    Thanks Mark - and the people you want to let go can be spottede even if you are using a wrong distribution (you might only need a transformation by a continous increasing function which will not change the ordering of the observation just the density).

  2. Joy, Thanks for your response to my blog. It's something that I really started thinking about this morning and I wasn't sure how well I had explained what was going on for me.

    The sigmas are really just a way of expressing which standard deviation you're referring to. First sigma is simply first standard deviation, second sigma is the second standard deviation and so on.

    Regarding the order versus the density, I think that's exactly the point. If you're in a dense population of data, the ordering doesn't really matter. What matters is which population you're in.

    Of course, there are always going to be challenges at the boundaries, the samples that could easily be in one standard deviation or another. But that aside, what matters is which group, not the order within the group.

    I'm really fascinated about the implications of bell curve distribution to relationships. For example, in terms of my beliefs, the way I want to live my life and my general philosophy of life, I feel that I'm way out of the tails of the curve. Depending on one's perspective, I might fall into the the leftmost tail or the rightmost tail, but nonetheless, the more I talk to "normal" people, the more I feel like I'm out on the edges.

    When I met Iris, I found someone who also lived on tails of the curve. Someone who shared the same unlikely beliefs, perspectives, wants, etc. For me, give that, everything else needed to create a great relationship came really easily.

    On the other hand, I have friends who are three sigma people in regard to their life views, their wants, etc. and they are partners with people who are one sigma people. They are constantly trying to reconcile all the differences that come from being completely different people. It turns out to be a lot of work.


  3. Right on Tef...I'm relating this to the past 3 group sessions at a church, for those dealing with "addictions"Doreen and I have been sitting in on, supporting a family member, found them wonderfully stimulating. We broke into small table groupings to discuss how relationship to 'tribe' mattered, being each asked our thoughts, I found myself expressing love, acceptance, back onto realizing, appreciating, every experience is simply a choice. In a way, I shared myself ok comfortable with being outside of the sheep mentality 'norm.' (Religions it seems to me frequently get distracted by their 'stuff,' and tend to complicate...)I loved sharing how it can be simple or complicated. Embracing 'tribe,' yet still alone with our choices, emphasizing the dynamic of being a free individual alongside of part of a 'herd.'

  4. LOL, in my head the ordering was more like: when do I want to go for the right tail?
    - my confusion came from wanting the sigma 3's which seemed to go left and rigth...
    arrhg I am working on a presentation on scoring models and I have to make it understandable for the coaches in the call center, and what does it help me to get confused? I'm not going for the sigma 3 in this situation, morelikely 1/2 sigma...

    If you want we can discuss when to use sigma's and when to use clusters next time we meet...

    Lots of love



Read, smile, think and post a message to let us know how this article inspired you...