In the federal government, the push to embrace data driven decision-making has gained serious momentum in recent years. From a people perspective, we’ve seen the establishment and hiring of Chief Data Officers across the federal government. From a policy perspective, H.R.4174, Foundations for Evidence-Based Policymaking Act of 2018 became law. The law laid the foundation for the Federal Data Strategy and 2020 Action Plan. From a process, partner and platform perspective, there are exciting new techniques and technologies that enable being data driven. (Table 1)
Table 1
Given that these people, policies, processes, partners, and platforms are in place it would seem we are data driven. There’s a significant upside to being data driven but equally significant pitfalls to doing it wrong. The data driven 2×2 matrix (Figure 1) highlights some of the problems and opportunities of being data driven. This article is the 1st in a 6-part series where we’ll discuss different aspects of the 2×2 matrix. Let’s begin by defining the mechanics of action of the 2×2 matrix.
Figure 1. Data Driven 2×2 Matrix.
Exploring the 2×2
The axises of the 2×2 matrix are our data and our perspectives. The y-axis ranges from the data you know about to the data you don’t (or didn’t) know. Your confidence in each data driven decision can only be as strong as your confidence in the data that informed your decision. However, most data lacks consistency, accuracy, and/or completeness. Mitigate this by profiling your data so you can determine your level of confidence. Profiling is the process of evaluating the consistency, accuracy, and completeness of your data. Once profiled, adjust your level of confidence, or data quality, accordingly.
On the x-axis is perspectives. After all, your perspective is your reality! You know what you know, and you don’t know what you don’t know. Your perspective also brings biases, both consciously and subconsciously. Mitigate this by expanding your perspectives. This axis ranges from the perspectives your team has to the perspectives they don’t have. Teams tend to have cognitive homogeneity which produces subjective thinking. Increasing cognitive diversity1 creates the potential for greater objectivity.
The Quadrants
Let’s explore each quadrant to better understand why there could be a problem with being data driven (Figure 2). Let’s start in the lower-left data-driven quadrant, small fish small pond. Work here is built with data you have access to from the perspective you have. This is analogous to fishing for few fish in a small pond. This produces limited insights & limited outcomes.
Figure 2. Fishing for answers.
There is a temptation to move to the upper left in search of more data. After all, more data means we’re data driven, right? Unfortunately, no. Moving into the upper-left quadrant often leads to data dragnet. Work here is built with all the data from the perspectives you have. This is analogous to fishing trawlers scraping the bottom of the ocean, they’re grabbing at everything hoping for something. The result – so much data it’s a struggle to find answers.
Conversely, instead of increasing your data what happens when you increase perspectives? Moving to the lower-right is the transparency and independence quadrant. Work here is built by increasing our perspectives on the associated data. Increase perspectives by expanding your team to include:
• cross-functional members
• other demographic groups
• people from outside your enterprise, especially those who might be affected by the results of your work.
Expanding your perspectives is like fishing with more fishing lines. The result – more insights & less bias.
Having increased our perspectives by moving from left to right, we are now in a position to selectively move upward to get targeted data. Moving to the upper-right quadrant, we have Nirvana. This is the ideal space to be in as the work here is with pertinent data and the broadest perspectives. This is like a team of spearfisherman going after specific fish. The result – more answers, better outcomes.
My personal
data driven journey
My journey began in 2015 with the standup of the Air Force Installation & Mission Support Center (AFIMSC). During an initial briefing with the new AFIMSC Commander and the Air Force Materiel Command’s Commander of the time, they highlighted the need to provide a status of all Air Force ranges. The standup and briefing led to a new opportunity of centrally managing all Air Force small arms firing ranges, an endeavor never achieved in USAF history. Specifically, they asked: If we had a dollar to spend, where should we spend it? Do we have a 1-N list, a prioritized list? If they spent that dollar, what would be the 2nd, 3rd, and 4th order effects of that decision?
The opportunity to centrally manage, and the line of questioning, laid the groundwork to pursue a data driven approach. As we progress through the 6-part series, I’ll walk you through my personal data driven journey as it relates to the 2×2 matrix.
What to do about it
So how can you reach Nirvana? Began by understanding where you are on your data driven journey. Here are steps you can take today:
• Validate your business/mission understanding: Ask yourself, “what critical business question(s) am I trying to answer?”
• Understand your perspective: “What do I know about the question? Do I understand what is driving the question? Who could offer a completely different viewpoint?”
• Identify your data sources: “What data do I know about that’s related to my question?
What data do I need but don’t have? How can confident am I in the data?”
• Build actionable insights: “Do I just have a stat or a performance metric? Does the metric reveal a material gap? Does the gap suggest actionable insights?”
Legend
Stat = averages, sums, counts etc.
Metric = a stat measured against a goal
Performance metric = goal achievement over time
The decisions you make based on your data and perspectives affect people’s lives. Your decisions also affect how you spend your precious dollars and ultimately how well you deliver your mission. The trick is not to over-emphasize the “data” in “data driven”. More data is not the goal. The goal is better decisions. And better decisions call for broadening your perspectives and selectively adding data. The data driven journey has much promise and hope. But if you’re not careful, you can get in trouble quickly. In our next article, we’ll take a deeper look at the 1st quadrant, small fish small pond.
Jim Eselgroth
Deputy, Chief Technology Officer
jeselgroth@definitivelogic.com
As Definitive Logic’s Deputy Chief Technology Officer, Jim helps government leaders optimize mission outcomes by leveraging technology, thought leadership, and change management techniques. Jim ensures excellence in the delivery of full life-cycle digital transformation activities, long-term technology strategy and vision planning, and innovation project portfolio management.
1 L. Hurley, B. S. Kristal, S. Sirimulla, C. Schweikert and D. F. Hsu, “Multi-Layer Combinatorial Fusion Using Cognitive Diversity,” in IEEE Access, vol. 9, pp. 3919-3935, 2021, doi: 10.1109/ACCESS.2020.3047057.
