Monday, September 7, 2009

Improving Your Intuition

In his book The Drunkard's Walk: How Randomness Rules Our Lives, author Leonard Mlodinow shows us how we often make mistakes that feel right simply because we don't understand probability and statistics.

On the one hand, I'm a big believer in the ontological, right-brained, going-with-my-gut approach to decisions and life. It's served me really well.

On the other, going with my gut often keeps me in undesirable situations longer than I would be were I to use my epistemological, left-brained, analytical side.

Positively Wrong
In one example, Mlodinow talks about Daniel Kahneman, a psychologist who won the 2002 Nobel Prize in Economics. Kahneman spent years studying and clarifying the types of misinterpretations of randomness that lead to common fallacies and false assertions.

In the mid-1960s, Kahneman was invited to lecture to Israeli air force instructors on the psychology of flight training. As he described how positive reinforcement works and negative reinforcement doesn't, one of the instructors interrupted him, disagreeing strongly.

The instructor pointed out that, when a student pilot has a really great day flying and receives praise for it, the student invariably flies less well the next day. Alternatively, if a student flies extremely poorly one day and is berated for it, the next day he will invariably fly better. Other instructors in the room chimed in voicing their agreement with the first instructor.

So, you have a situation in which the evidence clearly points to negative feedback working (i.e., absolutely every time that I berate someone for poor performance, they perform better), and positive reinforcement not working (i.e., every time I praise someone for exceptional performance, they regress.) The intuition of the instructors was that negative feedback works and their experience supported it.

What do you make of that?

Regression Towards the Mean
The thing that the instructors didn't understand was a statistical concept called regression towards the mean.

Every one of us, in every activity, has an average level of performance. If we run regularly, we run on average at a certain speed for a certain period of time. At work, we have an average level of productivity. Even our emotions tend to have an average daily state.

On some days, we really perform well. We feel really great and we run much faster and/or longer. We're exceptionally clear and our work productivity exceeds anything we've ever done. We wake up with an amazing sense of clarity and purpose and spend the day on an emotional high. Our performance is soaring.

On other days, every muscle in our bodies rebels, our breathing is shallow, and we barely finish our run. We daydream the day away at work. We find ourselves in an emotional funk that even five cups of coffee can't shake loose. Our performance sucks.

The thing is that both the high performance day and the low performance day are exceptional. Somewhere in the middle is the average or mean level of performance. Statistically, it's nearly impossible for us to follow an exceptional performance with an even better performance. Similarly, it's highly improbable that we'll follow a day of really bad performance with an even worse performance. In general, our performance tends to move or regress towards the average or mean.

Probability, Intuition and Attribution
Our Israeli flight instructors had experienced regression towards the mean in their students' performances. It was nearly impossible for a student to repeat or exceed an exceptional performance on the following day. It was also highly unlikely that a student would repeat an exceptionally bad performance.

The instructors' use of negative and positive feedback had very little if anything to do with it. It was just coincidental; yet they'd built a strong belief system about punishment and reward that seemed intuitive and was supported by the evidence.

Tuning Up Your Intuition
The Drunkard's Walk is full of examples of how we humans manage to successfully employ our intuition to fail tests, lose money, and generally make bad decisions. It also shows us that, by developing an understanding of randomness and probability, we can learn to make better decisions.

Whether you read the book or not, I'd like to invite you to engage the concept of regression towards the mean in your life. In general, if you want to develop a strong and accurate sense for how things are going (from working with your child to trying to lose weight to evaluating an employee to learning to play piano), you want to learn to discount the exceptions (bad days at school, increases in weight, stellar days on the sales floor, fat thumb Wednesdays) and look at how things are going on average.

Change your mean, change your life.

Have a great week!

No comments:

Post a Comment

Read, smile, think and post a message to let us know how this article inspired you...