Thought Leadership
No matter who you ask, there is rarely so much consensus in the business world as when it comes to the relevance of Generative Artificial Intelligence (GenAI). It will significantly shape the future of all companies.
This makes a look at the current reality all the more astonishing: many organizations are actually hesitant to implement (Gen)AI in their day-to-day work processes. What are the main reasons for this discrepancy between theoretical insight and practical procrastination?
In the Work Trend Index 2024 by Microsoft and Linkedin, 79 percent of managers in Germany rate AI as a key factor for competitiveness. So far so good. But the current situation paints a picture that cannot keep up with these claims.
A recently published Bitkom study, based on a representative survey of 603 companies, reveals that only six percent of German companies actually exploit the potential of their data. 60 percent do not exploit the possibility of using data at all or only to a limited extent. And data is the oil for AI. According to a recent survey by Dataiku, only a fraction of companies are currently succeeding in not only experimenting with GenAI, but also integrating it into everyday work processes on a large scale. Against this backdrop, the sobering results of a report by BARC and the Eckerson Group, supported by Dataiku, are not surprising. Only three percent of the 238 data experts surveyed worldwide, including managers and users, stated that GenAI is already fully integrated into their operational processes. In the aforementioned Work Trend Index, 55 percent of respondents lack a concrete objective for AI in their own organization and a resulting concept for implementation.
The costs of hesitation and procrastination are drastic. According to McKinsey, the economic value created by GenAI can amount to between 6.1 and 7.9 trillion US dollars per year worldwide across all industries through greater productivity and new or adapted use cases. Can – assuming the technology is actually used. So what is the current problem in many companies?
Technological hurdles
If you want to integrate AI, you need high-quality data – and GenAI is no different. Over time, companies have accumulated mountains of data in very different formats from very different sources. There is often a lack of systematic management and an understanding of who has what rights to access or process this data. The Bitkom study cited above illustrates the consequences of this bottleneck for GenAI in day-to-day business.
As with the implementation of “classic” AI use cases, the systematization of data is only the starting point for GenAI use cases. It therefore consists of a combination of data preparation/linking, (classic) modeling, the use of one or more large language models (LLMs) as well as the deployment and continuous monitoring of the use case. However, these aspects are often addressed using separate technologies – with consequences for the company: Linking different technologies leads to time delays in implementation, higher license and maintenance costs and frustration for those involved. For organizations that want to leverage the potential of their data through GenAI, there is no way around systematizing their data sets. However, the logic should be reversed – the systematization of data sets follows the prioritization of AI use cases. Otherwise, organizations will only be concerned with data quality for years without having implemented a single use case. In addition, it makes sense to use a platform that covers all aspects of implementing a (Gen)AI use case.
Economic hurdles
What does (generative) AI cost? Not a trivial question. In contrast to “traditional” AI use cases, where the majority of compute costs are incurred during the initial training of the models, the costs for GenAI use cases are mainly incurred after going live through the use of LLMs – regardless of whether they are (open source) LLMs running in the company’s own infrastructure (e.g. Llama 3 from the provider Meta) or licensed LLMs (e.g. GPT-4 from the provider OpenAI). One could even go so far as to describe “traditional” AI use cases as largely capex and GenAI use cases as largely opex. Capex refers to expenses for long-term investments that are incurred in traditional AI, especially during the initial modeling and training of the AI models. Opex is the ongoing cost of running and maintaining a business, which in the case of GenAI is incurred primarily after deployment through the continuous use of LLMs such as GPT-4 or Llama 3, as these require ongoing computing resources. And in the case of GenAI use cases, opex can increase unpredictably and rapidly. If you imagine a pharmaceutical company with 150,000 employees, 10 percent of whom work in sales and have recently started using a GenAI-supported customer approach, this can mean 10,000 LLM requests per day for this use case alone. With prices of 0.09 US dollars per 1K tokens (costs for input and output combined; 1000 tokens correspond to approx. 750 words) for GPT-4, for example, the costs seem “reasonably” manageable at first glance. In this case, they amounted to approx. 200,000 US dollars per year (0.09 US dollars x 10,000 LLM requests/day x 220 working days). However, this scenario only applies to a single use case and with adequate utilization. However, with hundreds of GenAI use cases and (unintentional) misuse, the costs can rise quickly. Provided the quality is comparable for the respective use case and the necessary infrastructure is available, switching to an open source model can significantly reduce costs for a large number of requests.
To avoid any nasty surprises, companies should ensure that they continuously record all costs that arise from the use of (Gen)AI. By testing and comparing different LLMs before a use case goes live, costs can ideally be kept in check right from the start. It should also be possible to centrally manage which employees can use which (Gen)AI use cases and from which amounts additional approvals are required. If, in addition to the costs, companies can also quantify the added value created by (Gen)AI, decision-makers have good arguments for expanding (Gen)AI to other areas of application.
Cultural and organizational hurdles
According to the survey by BARC and the Eckerson Group, why have only nine percent of companies already implemented GenAI in their operations or fully integrated it into their business processes? Beyond the question of cost, the uncertainty within the company is a major factor. Employees do not know how their daily work will change and what active contribution they can make. According to the Workforce 2.0 report by management consultants Mercer, 58% of managers surveyed are concerned that their company is not doing enough to get employees excited about new technologies. Two thirds fear that these will be implemented without work processes being adapted to them. In fact, employees outside the Analytics & AI expert teams are still involved too little or too late in the implementation of use cases – with the result that the new findings and processes are often not used by employees in the specialist departments.
It is particularly helpful if data experts and specialists without IT expertise, who in many cases ultimately use AI in their day-to-day work, work closely together as early as possible. In addition, there are numerous (Gen)AI use cases that specialists can implement independently with the appropriate training and tooling. This will get employees excited about the technology on a large scale and significantly more use cases will be implemented.
Legal hurdles and risks
GenAI use cases harbor a number of legal hurdles and risks, especially if they are used by external parties, i.e. outside the company. An apt example is that of a North American airline whose chatbot falsely promised a customer a discount. The company lost the legal dispute and had to pay compensation. This appears to be a classic case of AI hallucination – the LLM transmits (often with the greatest conviction) false or partially false information, which, however, often appears logical or possible in context. The generation of toxic or offensive content by the LLM is also considered a challenge – there is a risk of liability claims and a serious loss of reputation. In addition to the generation of false information (hallucination) and toxic information, there is a risk that LLMs will disseminate confidential content. This can happen both internally (for example, salaries that were part of the data provided to the LLM for an HR bot) and externally (for example, customer data available to an LLM-based marketing bot). It is therefore no surprise that the topic of generative AI is also addressed in the recently adopted EU AI Act as a classic example of a “general purpose AI model”.
Companies must ensure that their AI models do not generate false, toxic or confidential information, especially if the models are used by external parties. By 2026, companies must implement appropriate measures to minimize the risks of AI hallucinations and data breaches. Failure to do so could result in fines of up to 35 million euros or 7 percent of global turnover in the previous financial year.
Companies must manage the balancing act between agility and regulation. In addition to establishing and adhering to processes, it is essential to raise awareness and train employees on the risks and opportunities of (Gen)AI. The frequent fragmentation of processes and supporting software, for example between accessing and processing data on the one hand and training and monitoring models on the other, should be overcome and standardized. This will make the documentation of the entire value chain of a (Gen)AI use case more comprehensible, reduce the necessary resources and minimize the legal risk.
