Analytics and AI-driven enterprises thrive in the Age of With

More than a decade after the concept of big data became part of the lexicon, only a minority of companies have become insight-driven organizations—and culture may be the culprit.

The amount of data available to organizations every day continues to proliferate at a staggering volume. But technologies such as analytics and artificial intelligence (AI) have the potential to help businesses make better use of these massive volumes of data. In an age of collaboration between humans and machines—what we call the "Age of With"¹—organizations can gain advantage by designing systems in which humans and machines work together to improve the speed and quality of decision-making.

Learn more

Explore the Analytics collection

Download the Deloitte Insights and Dow Jones app

Subscribe to receive more content

But not every organization is optimizing the opportunities available in the Age of With. Some do little or nothing with data to aid their decision-making. Others carry out analytics projects in pockets of the business. Far fewer consistently embed analysis, data, and evidence-based reasoning into their decision-making process.

Most large companies fall into the last two categories, as do all the companies surveyed recently by Deloitte (see sidebar, “Survey methodology: Becoming an insight-driven organization”).

It’s understandable why many companies feel they are far down the path of becoming an insight-driven organization. Many have invested in creating the requisite data initiatives, analytics, or data science groups. Many have created chief data officer (CDO) or chief analytics officer (CAO) organizations. The vast majority have invested in tactical solutions.

These evolutions seem natural. It has been more than a decade since the term “big data” became part of the lexicon. Many legacy issues that traditionally posed barriers have now been eliminated or reduced. These include the high cost of data storage, expensive proprietary software, and the need to devote capital to expensive data centers.

But what is the reality? How many companies have truly evolved into insight-driven organizations?

To find out, in April 2019, Deloitte posed questions to more than 1,000 executives working at large companies (500+ employees) who interact with, create, or use analytics as part of their job. The goal of the survey was to see how many identified their company as being in the top two categories of the Insight-Driven (IDO) Maturity Scale (see figure 1)—analytical companies and analytical competitors²—as well as discovering how fully they leverage data and tools to make decisions, and what role culture and talent play in IDO maturity. Among our findings:

Most executives do not believe their companies are insight-driven. Fewer than four in 10 (37 percent) place their companies in the top two categories of the IDO Maturity Scale, and of those, only 10 percent fall into the highest category. The remaining 63 percent are aware of analytics but lack infrastructure, are still working in silos, or are expanding ad hoc analytics capabilities beyond silos.
Culture can be a catalyst or a culprit. Establishing a data-driven culture is harder than acquiring the right tools or hiring the right talent. But it pays off. Organizations with the strongest cultural orientation to data-driven insights and decision-making were twice as likely to have significantly exceeded business goals. Among the 37 percent of companies in the survey with the strongest analytics cultures, 48 percent significantly exceeded their business goals in the past 12 months, making them twice as likely to do so compared to the 63 percent that do not have as strong an analytics culture.
Aim high for analytics champions. Executive sponsorship is vital to this level of organizational change and the best champion sits in the corner office. According to the survey, the CEO is the lead champion of analytics in 29 percent of companies surveyed, and these companies are 77 percent more likely to have significantly exceeded their business goals. They are also 59 percent more likely to derive actionable insights from the analytics they are tracking.
Most executives are not comfortable accessing or using data. Fully 67 percent of those surveyed (who are senior managers or higher) say they are not comfortable accessing or using data from their tools and resources. The proportion is significant even at companies with strong data-driven cultures, where 37 percent of respondents still express discomfort. This points to a major opportunity for companies to provide more education and improve the user experience if they want every employee to use insights as part of their work.

The Insight-Driven Organization Maturity Scale

Survey methodology: Becoming an insight-driven organization

To obtain a cross-industry view of how organizations approach business analytics and where they fall along an analytics continuum, Deloitte conducted an online survey of 1,048 executives (senior managers or higher) who interact with, create, or use analytics as part of their job and work for US-based companies with 501+ employees. Thirty-six percent are senior vice-presidents and above, and 13 percent are from the C-level. We eliminated 928 potential respondents representing approximately 35 percent of the potential survey base because they do not have any interaction with analytics in their companies. The survey was fielded in April 2019 with a margin of error of ± 3.03 percent at the 95 percent confidence level.

Business analytics becoming mainstream

Three-quarters of survey respondents report that their organization’s analytical maturity has increased over the past year, and nearly as many—70 percent—expect business analytics to be more important in the next three years than it is now.

Accompanying these indicators of increased organizational awareness is the finding that over the next few years, business analytics as an organizational priority is expected to be on par with such critical drivers of business value as risk management, reputation management, product/service innovation, and managing growth expectations (see figure 2). In other words, analytics is becoming an established fact of business life and no longer the sole domain of the IT or finance department.

AI, the more technology-intensive relative of business analytics, is not yet used as commonly as some other business and management tools—though other Deloitte surveys suggest that its use is growing even more rapidly than business analytics.³

The importance of business analytics is rated close to reputation and risk management

As figure 3 shows, analytics is being used to support a number of strategic areas representing a broad spread of fairly mainstream use cases with no low frequencies. The top use—identifying business process improvements—aligns with one of the top organizational priorities among respondents shown in figure 1—overall cost efficiency. Using analytics to improve processes, such as optimizing pricing or inventory levels, represents a classic use of analytics.

Slightly less common is the use of analytics to guide product and service development or monitor competitors. This is not surprising. Typically, there is less structured numeric data available in these functions. Also, as is discussed later in this report, companies are far less likely to rely on unstructured data such as product images or customer comments gathered from interacting with a chatbot while shopping online.

Identifying business process improvements and understanding and improving customer experience top the list of use cases for analytics

The insights trifecta: Data and tools, talent, and culture

Being an insight-driven organization is not the result of any single factor; it is multidimensional. For organizations to fully leverage the insights they derive and embed them into decisions and actions, a combination of three drivers is required: data and tools, talent, and culture.

Beyond BI and spreadsheets

Although it’s only one component of success, investment in and adoption of data and tools is perhaps the most visible and easily measured manifestation of where a company is on its journey to becoming an insight-driven organization.

The traditional workhorses of the data analytics universe—spreadsheets such as Microsoft Excel and business intelligence tools such as Microsoft Power BI or IBM Cognos—are the most commonly used tools. Yet 67 percent of the companies surveyed also use at least one advanced tool such as SAS, an open source tool such as R, a programming language such as Python, or an AI tool.

67% of the companies surveyed deploy at least one advanced tool to access and generate business analytics

This range of tool sophistication is encouraging. However, most organizations in the survey are at risk of hindering the success of their analytics by limiting the types of data they employ and limiting the chance to adopt a cross-enterprise approach.

According to our survey, most organizations (64 percent) today report relying on structured data from internal systems/resources. Far fewer (18 percent) have taken advantage of unstructured data (such as product images or customer audio files) or comments from social media. These data types can be difficult to put in the row-and-column relational format characteristic of traditional data storage, but over the last decade several new technologies have emerged to address that, including Hadoop and other open source projects, cloud-based architectures, approaches to managing streaming data, and new storage hardware environments.⁴

These forms of data are often more challenging to interpret but can deliver a more comprehensive and holistic understanding of the bigger picture—particularly of the world outside an organization. A look at companies that exceeded their business goals shows a connection with this appreciation for unstructured data: Executives who say unstructured data is one of the most valuable sources of insights are 24 percent more likely to have exceeded their business goals, according to our survey.

The majority of companies today also adopt a fragmented, siloed approach to analytics tools and data, which correlates with diminished business success. Sixty percent of companies in the survey that use different tools and systems across different teams or business units exceeded their goals last year. But among a much smaller group—the 26 percent that use a single, common set of tools and methods across the enterprise for accessing and analyzing data—an impressive 80 percent exceeded their business goals last year. The absence of one enterprise approach to finding insights is a common barrier to effectiveness.

Data terms decoded

Analytics: The systematic quantitative analysis of raw data to draw conclusions that help drive business strategy and performance.⁵

Artificial intelligence (AI): The development of computer systems that can perform tasks previously performed only by humans.

Big data: Big data is structured and unstructured data generated from diverse sources in real time, in volumes too large for traditional technologies to capture, manage, and process in a timely manner.⁶

Machine learning (ML): A method of data analysis that automates the building of analytical models. These algorithm-based models are primarily built from statistical techniques and theoretical computer science and leverage large datasets to continuously learn and improve.⁷

Case study 1: A common data and analytics environment at Procter and Gamble

Procter & Gamble (P&G) has long maintained a common, enterprisewide approach to data management and analytics as part of its Global Business Services organization. It established global standards for data type and quality; with these standards, aggregating and comparing data across product lines and regions became significantly less complicated with the data being stored in standard formats in the central data warehouse. Over time, managers found that the single, companywide database played a much more strategic role in aiding decision-making by serving as the “one truth” for the entire corporation. The company’s IT organization also automated the generation of reports that were used across multiple business units. In addition to simplifying the task of retrieving data and performing some basic analyses, the reports served to standardize the way data was visualized across the company.⁸ With consistent visuals, analysts and managers from one unit could step into a role, or even a meeting, with a different product or region and quickly understand the situation. The IT organization also developed scaled and standard solutions for accessing analytics, including the Decision Cockpit, a web-based customizable dashboard that tracked the most relevant data and news for each individual employee. For management teams, P&G developed a patented physical environment for information-based executive decision-making called the Business Sphere. Leadership teams would typically gather in over 50 Business Spheres around the world at regular intervals to review product and market performance for the previous periods and make decisions about how to proceed going forward.⁹

Data science is a team sport

Talent can be spread broadly across an organization or concentrated among a select few. D.J. Patil, appointed as the US government’s first-ever chief data scientist in 2015, liked to say that data science is a team sport.

But if data science comprises teams within organizations, the teams are generally small and homogeneous. The survey data shows that two-thirds of organizations rely on a select group of employees who have been trained on analytics or data science, versus 27 percent who say that all employees are trained (quite rare in our experience) or the small minority that trains no employees.

Organizations need to embrace a diversity of roles and skills. Instead of relying on siloed teams of highly technical quantitative experts, companies would do well to cultivate a wide variety of people throughout the organization who are curious, numerate, and capable of translating between analytics/data science methods and business requirements. This might be called the “democratization” of data science.”¹⁰

Data from the survey confirms the merit of this approach. In companies where executives report that all employees have been trained on analytics, 88 percent exceeded business goals, compared to just 61 percent of companies in which only select employees have been trained on analytics.

Concomitant with the need to involve all employees in the use of analytics for decision-making is the need for user-centered design and stakeholder analysis. Decision-makers, data scientists, and business analysts, all must care about the business outcomes and be consulted frequently on analytics projects. Effective users of analytics are those who understand what matters most, prioritizing the right questions and zeroing in on the right decisions to improve. They need to understand where to find the insights and quickly know what to discard, to avoid being overwhelmed by the massive volumes of data that businesses now routinely generate or receive.

The issue of improving companywide access and use of analytics is not restricted to employees at lower levels in the organization. The survey shows that 67 percent of those surveyed (who are senior managers or higher and interact with data) say they are not comfortable accessing or using data from their tools and resources. Surprisingly, comfort level rises with title level, which could show access to resources also rises with level in the organization. In any case, the fact that two-thirds of executives at large organizations are not comfortable navigating what is now the lifeblood of every business is a gap that could yield huge dividends if addressed.

67% of our respondents aren’t comfortable accessing or using data from their tools and resources

Case study 2: Building an analytical talent ecosystem at Principal

Principal Financial Group, based in Des Moines, Iowa, offers insurance, retirement, and asset management products. Recruiting data scientists and AI specialists is difficult anywhere, and especially away from hotspots such as the Bay Area, Boston, and New York; so Joseph Byrum, the company’s chief data scientist, has been forced to be creative about building a talent ecosystem. In addition to hiring data scientists, he’s explored crowdsourcing and close relationships with university analytics programs. Over the course of his career, Byrum has worked with thousands of crowdsourcing projects, and he strongly believes that they generally yield more innovative solutions than working with internal domain experts. He works with several different crowdsourcing firms. In addition to crowdsourcing, Byrum and Principal also rely heavily on an even less common talent source—student capstone projects in analytically oriented programs at universities. There are now hundreds of such programs in the United States alone,¹¹ and many of them have “capstone” programs in which students are expected to work on real business problems with corporate sponsors. Byrum has set up over 60 capstone projects at a variety of universities—many involving important problems for Principal to solve. Of course, Principal does need to hire some internal people as well to help develop AI and analytics solutions, and Byrum has thus far built a team of about 20. He hires often through the capstone projects, which Byrum finds much more useful than interviews. He says that the staff that he does hire are a little different than most data science types in other companies—they have quant skills, but also design thinking and systems engineering skills. He also puts a strong emphasis on their ability to communicate and collaborate and uses psychological profiles to ensure that members of his team complement each other.¹²

A culture that acts on insights

Among the key drivers that help companies scale from carrying out analytics projects in pockets to becoming an IDO, a data-driven culture is the most difficult to establish. It also appears to be the one factor holding back many organizations.

A data-driven culture is one in which important decisions are made based on data and analytics (assuming that data is available). It is important to note that data, technology, and solid statistical and machine learning capabilities are facilitators but not necessarily drivers of insight-driven decision-making. There has to be a willingness to act on analytically derived insights—to make decisions, change processes, and adapt behaviors based on insights rather than intuition.

The survey highlights this powerfully by revealing a strong correlation between culture and business performance: Organizations that reported having the strongest cultural orientation to data-driven insights and decision-making are twice as likely to have reported exceeding business goals in the past 12 months. Forty-eight percent of these businesses say they outperformed their target versus just 22 percent of those with a more diluted analytics culture (see figure 4).

A twofold difference in not just meeting but exceeding business goals is a significant difference, and it contributes to the dividing lines between the insight haves and have-nots we describe in the introduction.

The link between insight-driven culture and business performance

Creating any form of desired culture is a challenge for most organizations, and analytics is no exception. Only 39 percent of respondents say their company has a strong cultural orientation to data-driven insights and decision-making, and a similarly low percentage (37 percent) feel employees in their company are aware of the importance of data analytics.

But, as with other areas in this survey, organizations that have a strong analytics culture and employee awareness of analytics are more likely to have exceeded business goals, illustrating the bottom-line benefits of investing in creating the right culture and elevating employee awareness of the importance of analytics.

How companies assign responsibility for analytics is a critical factor in moving up the maturity scale. Here the results are encouraging: Analytics is deployed companywide at the majority of organizations (57 percent), while one in three deploy across specific company groups and only one in 10 deploy them in an ad hoc manner.

As with talent, the survey data indicates diffusing responsibility across organizational lines is more effective than localized responsibility (see figure 5). However, the survey shows most rely on a pool of identified experts rather than charging all employees with responsibility for developing insights or learnings from data analytics. In other words, analytics is not yet a team sport.

Broadest responsibility for analytics correlates most with exceeding business goals

The concept of necessary-but-not-sufficient is a critical one for insight-driven maturity. It is easy to mistake the necessary for good enough; but building tools and acquiring data are not sufficient to reach the top levels of IDO maturity. As figure 5 illustrates, spreading responsibility for analytics across the enterprise and making analytics-gathering and decision-making a team sport are crucial to success. Acquiring tools without engaging teams will not lead to better business outcomes.

The IDO Maturity Scale: Five kinds of organizations

As stated earlier, when companies in the survey were asked to rank themselves on the IDO Maturity Scale (see figure 1), 63 percent placed themselves in the bottom three categories—analytics aware, localized analytics, and analytics aspirations, meaning they are not insight-driven organizations (see figure 6).

The methodology used in the survey eliminated approximately 35 percent of the potential survey base because they did not have any interaction with analytics in their companies. This means the insight-driven organizations in the survey make up 37 percent of a group that is, at a minimum, analytics aware. Given that the overall population of companies includes those that are not even analytics aware, the number of insight-driven organizations in real life is very likely even smaller than this survey indicates.

In addition to measuring where companies placed themselves on the maturity continuum, we also wanted to examine what patterns emerge when we looked at how they approach analytics culture, data and tools, and talent. To analyze this, Deloitte cross-referenced respondent’s self-ranking on the continuum with other factors.

In general, development in key input (tools, talent, culture) and outcome (business success) metrics rises across the maturity continuum.

An interesting exception is the group that is lowest on the continuum, those that are analytics aware. They rated themselves higher (close to the top performers’ rankings) in all the key factors making up analytics maturity. Given that they represent only 6 percent of respondents, this anomaly represents a very small percentage of the data. It is possible that their atypical responses may be a layman’s appraisal of the factors driving analytics maturity. In other words, they may be an example of the adage, “You don’t know what you don’t know.”

The otherwise steady progression of best practices on the continuum below illustrates the importance of leadership and culture in becoming an insight-driven organization. These two factors are most likely to be the drivers of this segmentation, in our experience. For example, committed leaders can provide data and tools, and hire the necessary people.

How companies approach analytics: The Insight-Driven Organization Maturity Scale

Making culture a catalyst

In our experience—reinforced by this survey—the vast majority of companies do not have initiatives in place to address data-driven culture issues. Initiatives around data and technology, and even hiring specialized talent, are not enough to bring about the cultural changes needed to help companies evolve to being insight-driven.

Here are some recommended steps to bring about the cultural changes needed to become an IDO:

Hire or promote leaders with a strong orientation to analytics-based strategy and competition.
Educate employees at all levels and in all functions about the role of analytics in business decision-making.
Implement individual performance assessment tying the use of analytics to incentives.
Encourage leaders to model examples. In meetings, for example, demonstrate asking for data points to back up business decisions.
Make it easy for employees to act on data and analytics through nudges, an effective way to motivate desired actions.¹³
Use social proof, a concept explored by the prominent social psychologist Robert Cialdini and a standard part of the “nudge” toolkit, to inspire action by discussing how other companies are doing this.¹⁴
Reward trying and risk-taking, even if efforts fail. Create a culture that respects the notion of honorable failure.
Know the limits of analytics: If you can’t get the data, you can’t gain the insights.
Enlist an executive sponsor, ideally the CEO, so that you can melt away the “permafrost” of change-resistant middle management.

Final thoughts

Our 2019 survey results clearly show insight-driven organizations represent a minority of businesses today, despite the number of years technologies such as big data and analytics have been readily available. The most common culprit is culture. Buying and using analytics tools is not hard—changing behaviors is. By emphasizing education, enlisting executive sponsors, and modeling and rewarding the right insight-driven behaviors, organizations doing business in the Age of With can benefit from human collaboration made greater with AI and analytics.

Acknowledgments

Endnotes

Topics in this article

Deloitte Analytics and AI

Achieving your business outcomes, whether a small-scale program or an enterprisewide initiative, demands ever-smarter insights—delivered faster than ever before. Doing that in today's complex, connected world requires the ability to combine a high-performance blend of humans with machines, automation with intelligence, and business analytics with data science. Welcome to the Age of With, in which Deloitte translates the science of analytics—through our services, solutions, and capabilities—into reality for your business.

Learn more

Get in touch

Tim P. Smith
Principal, Strategy & Analytics
Deloitte Consulting LLP
[email protected]
+1 571 331 5867

DELOITTE INSIGHTS

DELOITTE RESEARCH CENTERS

Welcome!

Latest Insights

Recommendations

About Deloitte Insights

Topics for you