The Pessimist on Your Team

Pessimists have a bad reputation.

Today I came across an article called How to Handle the Pessimist on Your Team and it made me think how things can go really bad when you are running a complex project without being regularly subjected to healthy scepticism. A little later in the day I read Paul Krugman's How Did Economists Get It So Wrong? which described how the weight of opinion of economists had drowned out rational scepticism about the "efficient market" theory.

Weight of opinion is a powerful force and when the majority of people in a group believe something is correct it can be a bit rough for the minority who don't share that belief.

I once worked at a software company where some popular project managers told developers who did not believe their project schedules that they were being pessimists and they should cheer up. These project managers followed most of the strategies from How to Handle the Pessimist on Your Team and added a few of their own. They were pretty sweet about it and everybody including the pessimists ended up liking them. They got great results as well. Hard work, personal charm and genuine consideration for others, all of which these guys had, are stronger forces than the things I am discussing in this post.

After a few projects with these guys I started recording people's estimated completion dates at the start of projects and found that everyone was under-estimating the dates (the customer deliverables were in reasonable shape through schedule padding and some pragmatic last minute feature dropping). No surprises there, except that absolutely everybody including the "pessimists" underestimated the schedule. The interesting thing, and the point of this post, was how the estimates of the "pessimists" compared to the estimates of everyone else. The pessimists were the people whose estimated schedules were later than the consensus schedule estimate.

My explanation for this at the time was that everybody was trying to estimate the schedule accurately and fit in with the rest of the team as well. Those who valued giving accurate estimates over fitting in with the team gave estimates closer to the correct date and further from the consensus date. Perceived pessimism correlated with divergence from the consensus schedule and was largely independent of the true schedule.

That reminded me of how NPR thought it was being unbiased when it was in the middle of American mainstream opinion which was roughly halfway between Republicans and Democrats. This automatic reinforcement of the consensus view inhibits clear thinking. If Colombus had been an NPR journalist he would have accepted the consensus view of his time that the Earth was flat and not sailed to the Carribean.

Reinforcement of a core set of beliefs over time is one of the way organisations evolve. That is just how things are. Like most people I find myself drawn to organisations who share my core beliefs. However I find this tendency works against me when I need to be objective, such as when I need to meet schedules for critical projects involving many unknowns. In these cases, when nature cannot be fooled,  I know of no substitute for people who speak their minds about what I am doing wrong rather than tell me what I want to hear. Steve McConnell created some procedures for doing this that I have been using for over 10 years: the invaluable Top 10 Risks List, anonymous reporting channels (I prefer people trusting me that I appreciate them telling what I am doing wrong) and objective metrics from automated testing.

I have spent my most of my working life at companies that have encouraged a diversity of opinion and the people I work with are usually comfortable with practices that attempt to discover objective truths, like the Steve McConnell procedures. In many cases they have wanted to know what they were doing wrong so they could improve. However I have also regularly encountered people in nominally technical roles who didn't like the "bad news" these procedures revealed.  So, in my experience, there are still some technical managers out there trying to manage perceptions instead of managing reality and leaving the perceptions to the marketing folks. That's why I get suspicious about the motives of technical managers who talk about managing the pessimists in their teams.