Patterns and anti-patterns to data driven decision making

As leaders, we’re often under immense pressure to make decisions. The higher up in the organization we are, the more the pressure and consequences are magnified. The more we’re removed from the tactical elements of the organization, the less raw information we have access to. To compound matters, the information being presented has been filtered and a tailored message accompanying it. And if we’re in an organization that only focuses on what’s good, we’ll rarely hear about the bad until it’s a catastrophe. Becoming a data driven organization can help. So, what can you do? Well, there are patterns and anti-patterns to being data driven.

To bring yourself up to speed please read:

Part 0
Part 1
Part 2
Part 3
Part 4

Figure 1 – Problem with being data driven matrix

So far in this 6-part series, we explored the 2×2 matrix (Figure 1), the problem with being data driven. In this final article, we’ll explore the patterns and anti-patterns of being data driven.

Interpreting data requires that you look at the whole picture, what’s there and what’s not there. For example, during World War II, the statistician Abraham Wald took survivorship bias into his calculations when considering how to minimize bomber losses to enemy fire (Figure 2). The Statistical Research Group (SRG) at Columbia University, which Wald was a part of, examined the damage done to bombers that had returned from missions and recommended adding armor to the areas that showed the least damage.

This contradicted the US military’s conclusion that the most-hit areas of the bomber needed additional armor. Wald noted that the military only considered the bombers that had survived their missions – ignoring any bombers that had been shot down or otherwise lost, and thus also been rendered unavailable for assessment. The bullet holes in the returning bombers represented areas where a bomber could take damage and still fly well enough to return safely to base.

Figure 2 – The damaged portions of returning planes

Therefore, Wald proposed that the Navy reinforce areas where the returning bombers were unscathed, inferring that bombers hit in those areas were the ones most likely to be lost. In this example, the anti-pattern is making the decision based solely on what you see, only those bombers that survived. Instead, mitigate survivor bias and consider all potential instances. The pattern is seeing the whole picture and making the decision accordingly.

A great way to look for what is there and what is not there is to ask critical thinking questions. Here are a few that you might find helpful when assessing your data:

…benefits from this?
…could this be harmful to?
…makes decisions about this?
…would this cause a problem?
…can we expect this to change?
…will we know we’re successful?
…is another perspective?
…is another alternative?
…is the most / least important?

…is this a problem / challenge?
…is it relevant to you / others?
…is this the best / worst scenario?
…will this idea take us?
…do we go for help with this?
…can we get more information?
…does this disrupt things?
…will we approach this safely?
…do you prioritize these against each other?

There’s an old joke about a patient saying to a doctor, “It hurts when I do this”. The doctor replies, “Don’t do that.” We already know there are several anti-patterns to ‘not do’. Other people made these mistakes…so you don’t have to. I’ve captured some of them in the table in Figure 3. Skip the anti-patterns. Apply the patterns. You’ll be glad you did.

Finally, culture is probably the most vital element of becoming a successful data driven organization. A weak culture avoids transparency. People don’t trust. Issues are hidden. On the other hand, a strong culture is transparent, has trust, and issues are brought into the light. It takes everyone but starts with leadership. The leaders throughout the organization, as proven by their actions, will show if they are transparent, that they trust, and want to hear about issues. You’ll know this because:

To be transparent, the leader is open about how decisions are made

To have trust, the leader does what they say, they’re approachable, and support the team both when they’re successful and make mistakes

To hear issues, the leader reacts in a positive way; they thank you for the information

Notice no technologies are listed above, they’re all soft skills. You can have the best technical and data architecture in the world and still fail to make the right data driven decision. If the culture is right, all the people interacting with the architecture will help improve it. This culture encourages organizations to reach Nirvana because people bring more perspectives and the relevant data. If the culture is weak, they’ll do the opposite.

As a leader, you’re faced with such immense pressure to decide. Help yourself make great decisions. Be aware of survivor bias; look at what data is there and not there. Ask critical questions, so you understand the data. Adopt the patterns and avoid the anti-patterns. Finally, and most importantly, invest in your organization’s culture. “The stronger the culture, the less corporate process an organization needs.” – Brian Chesky

Jim Eselgroth

Jim Eselgroth

Deputy, Chief Technology Officer

As Definitive Logic’s Deputy Chief Technology Officer, Jim helps government leaders optimize mission outcomes by leveraging technology, thought leadership, and change management techniques. Jim ensures excellence in the delivery of full life-cycle digital transformation activities, long-term technology strategy and vision planning, and innovation project portfolio management. 

Suggested content for you

How Definitive Logic’s Financial Management practice can help Federal agencies transform their financial processes

How Definitive Logic’s Financial Management practice can help Federal agencies transform their financial processes

Are outdated legacy systems, manual processes, compliance requirements, and resource allocation challenges preventing your agency from achieving its...

read more