Detecting Bias in Your Decision-Making

Warren Buffett is regarded as one of the most successful investors in history.  He and his partner, Charlie Munger, attribute a large part of the success of Berkshire Hathaway to the partnership’s ability to make investment decisions without the influence of cognitive bias that risk every human decision.

Whether it’s about investments, business strategy, political candidates, or personal matters, we all try to make good decisions. Unfortunately, emotion and bias is part of human psychology.  While we can’t eliminate bias completely, we can each develop our own toolkit for detecting and mitigating against those unhelpful mental quirks that can lead us down the wrong path if we’re not careful.

Paul Graham of Y Combinator has written a thoughtful essay describing an elegant but subtle method of detecting bias in the evaluation of applicant pools.  The interesting idea in Graham’s observation is that it allows third-parties to detect bias in an organization’s decision-making, even if that organization makes efforts to screen certain details of its process.

Graham suggests that bias can be detected whenever “(a) you have at least a random sample of the applicants that were selected, (b) their subsequent performance is measured, and (c) the groups of applicants you’re comparing have roughly equal distribution of ability.”  Graham explains that in these circumstances, bias can be measured by comparing the back-end success of different groups of applicants, even if you cannot view the applicant pool itself:

How does it work? Think about what it means to be biased. What it means for a selection process to be biased against applicants of type x is that it’s harder for them to make it through. Which means applicants of type x have to be better to get selected than applicants not of type x. Which means applicants of type x who do make it through the selection process will outperform other successful applicants. And if the performance of all the successful applicants is measured, you’ll know if they do.

Graham provides a helpful example of detecting gender bias in the venture capital world:

For example, many suspect that venture capital firms are biased against female founders. This would be easy to detect: among their portfolio companies, do startups with female founders outperform those without? A couple months ago, one VC firm (almost certainly unintentionally) published a study showing bias of this type. First Round Capital found that among its portfolio companies, startups with female founders outperformed those without by 63%.

Graham’s idea seems applicable to any process through which various individuals or opportunities are screened for participation or selection through some pre-defined criteria.  This could include hiring decisions, investment decisions, account or client decisions, or media or networking opportunities, just to name a few.  If you find that a certain group of applicants, or investments, or account type is outperforming the average of the total selected pool, you may have revealed some cognitive bias in your process disposed against the higher-performing group.

Improving decision-making requires a constant eye scanning your processes for places where bias might hide, and from which it might rise up to influence a decision.  Couple Graham’s idea with Shane Parrish’s The Work Required to Have an Opinion and Musashi’s tactics for understanding the strength and weakness in your position.

One thought on “Detecting Bias in Your Decision-Making”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s