17 September 2024

Generative AI faces challenges in risk and data handling, new Deloitte study shows

Despite growing investment and initial enthusiasm, risk management and data handling remain key challenges to the wider use of Generative Artificial Intelligence (GenAI).

AI theme

The third quarterly Deloitte AI Institute™ State of Generative AI in the Enterprise report shows that two-thirds of organizations (67%) are increasing investment in GenAI because of its proven potential. However, the report also highlights that barriers such as data and risk management issues are preventing full development and dampening the enthusiasm of executives. It shows that demonstrating the value of GenAI deployment and overcoming the challenges of integrating it effectively into organisational structures will be key to continued investment.

The Deloitte AI Institute™ released the third quarterly edition of its State of Generative AI in the Enterprise report. It provides an overview of the current state of GenAI adoption and usage, and reveals how organizations are overcoming obstacles and achieving measurable results using the technology. "The State of Generative AI in Companies" is based on a survey of 2,770 director- and executive-level respondents in 14 countries. Although respondents report varying levels of familiarity with generative AI, all have experience with the technology and are actively testing or implementing it in their organisations.

Key excerpts from the report

"As the benefits from the first successful experiments and use cases begin to emerge, a pivotal period for generative AI begins, balancing high expectations from executives with challenges such as ensuring data quality, managing investment costs, effectively measuring outcomes, and adapting to a changing regulatory environment. The Q3 survey shows that effective change management and deep integration of this technology into the fabric of organizations is now key to overcoming these obstacles and realizing the full potential of generative AI," said Jim Rowan, principal and leader of Applied AI at Deloitte Consulting LLP.

"We continue to see enthusiasm for GenAI across organizations, and leaders are able to make the most of this technology by integrating it into critical business functions and processes. Our research shows that the key benefits of GenAI go far beyond simply increasing efficiency, productivity and reducing costs. More than half of respondents highlight benefits such as greater innovation, improved products and services, or stronger customer relationships. The wide range of areas in which GenAI has benefited demonstrates the enormous potential and versatility of this transformative technology," says Costi Perricos, Head of Generative AI at Deloitte Global.

Enthusiasm is tempered by reality - and measurable examples of use cases will lead to successful scaling up

According to the survey, executives and board members continue to place their hopes in GenAI, but are now approaching it with more caution. Interest remains "high" or "very high" among 63% of senior executives and 53% of board members. However, these figures have fallen since the Q1 study, by 11 percentage points for managers and 8 points for executives. Although the goal is to select and rapidly develop GenAI projects with the highest potential, many initiatives still remain in the pilot or testing phase. The majority (68%) of respondents have so far put into practice no more than 30% of their attempts to use GenAI.

Executives focus on data lifecycle management as a foundation for GenAI deployment

Data is becoming a core issue for leaders who are embracing AI, with 75% of organizations increasing their investment in data management as they leverage GenAI. However, as organizations look to expand their use of GenAI, unexpected barriers are emerging - data-related issues led 55% of organizations surveyed to avoid some GenAI use cases. Addressing data vulnerabilities appears to be a key step in meeting GenAI's specific data governance requirements. Organizations are therefore strengthening data security (54%), improving data quality assurance practices (48%), and updating data governance frameworks or creating new data policies (45%).

Mitigating risks and preparing for regulation

Although respondents recognize that managing the risks associated with GenAI is key, three of the top four most frequently cited barriers to successful GenAI deployment are risk-related, including concerns about regulatory compliance (36%), difficulties in managing risk (30%), and lack of a governance model (29%). These concerns likely stem from specific risks associated with GenAI, such as model bias, incorrect results ("hallucinations"), new privacy concerns, trust, and protection of new areas of vulnerability. To help build trust and ensure responsible use, organizations are working to build new controls and oversight tools. The most common steps include establishing a governance framework to manage the use of GenAI tools and applications (51%), monitoring regulatory requirements and ensuring compliance (49%), and conducting internal audits or tests of GenAI tools and applications (43%).

As experimentation continues, there is a growing need to demonstrate the value of GenAI initiatives

Moving beyond the pilot phase, 41% of companies surveyed have difficulty accurately defining and measuring the impact of their GenAI initiatives. Only 16% regularly report to CFOs on the value that GenAI creates. As applications and use cases mature, leaders will be less willing to invest based solely on ambitious visions and fears of missing an opportunity - making measurement a key factor in maintaining interest and support from executives and boards. Organizations are using specific KPIs to demonstrate value (48%), creating a framework to evaluate GenAI investments (38%), and tracking changes in employee productivity (38%).

About the State of Generative AI in the Enterprise

The wave three survey covered in this report was fielded to 2,770 director- to C-suite-level respondents across six industries and 14 countries between May and June 2024. Industries included: Consumer; Energy, Resources & Industrials; Financial Services; Life Sciences & Health Care; Technology, Media & Telecom; and Government & Public Services. The survey data was augmented by additional insights from 25 interviews with C-suite executives and AI and data science leaders at large organizations across a range of industries. This quarterly report is part of an ongoing series by the Deloitte AI InstituteTM to help leaders in business, technology and the public sector track the rapid pace of Generative AI change and adoption. The series is based on Deloitte’s State of AI in the Enterprise reports, which have been released annually the past five years. Learn more at deloitte.com/us/state-of-generative-ai.