Customer service wants to automatically summarise customer conversations, while finance wants to extract and verify information from incoming documents. More and more departments are identifying opportunities for AI. These days, an initial pilot can be built quickly. “If a tool’s building blocks are already in place, you can set up a new use case in as little as two hours,” said Skip van der Meer, Managing Business & AI Consultant at Yellowtail Conclusion, during AI on Stage. At this event on 29 September, Conclusion brought together organisations from different sectors to discuss AI strategy and responsible innovation.
The fact that building is quick does not mean that implementation will be successful. Many organisations still struggle to embed a new AI application in their working processes. The bottleneck is therefore no longer the process of building the tool itself. So where are the real obstacles? Across the sessions and round-table discussions at AI on Stage, the answer became increasingly clear: the bottleneck lies in the decisions surrounding the technology. Read on to discover which decisions matter.
Bottleneck 1: The objective is not clear enough
‘Is IT really the bottleneck?’ This question was central to the session on value creation and strategy. When building applications still took months, many questions were answered along the way. What problem should the application solve, and is AI actually the right means of doing so? What should it deliver, and for whom? How do you measure that? What happens in exceptional cases? “Building is getting faster and faster, but making these decisions takes just as much time as it did before,” said Sophie van Klaveren, Business Consultant at Yellowtail Conclusion. “So make sure you ask the right, incisive questions in advance.”
You cannot answer those questions alone. Several departments across the organisation are involved, each with different interests. “The business wants results, Risk & Compliance wants assurance, and operational teams want a workable solution. IT cannot make these decisions on its own,” said David Buschman, Commercial Director at Conclusion Intelligence. “A successful AI application will only be adopted if someone takes the lead. That involves more than expressing enthusiasm or making budget available. Management must bring the right parties together, clarify what the application should contribute to and make decisions that would otherwise remain unresolved.
Bottleneck 2: When is it good enough?
Conventional software produces the same output for the same input every time, but an AI application based on a language model does not. Answers may vary and are not always correct. As a result, ‘it works’ is no longer the end point. You need to decide for yourself how well the application should perform, and that proves more difficult than it sounds. What is good enough?
An example from the event’s joint opening session makes this concrete. One of the companies attending the event built a chatbot for its internal knowledge base, which raised a new set of questions. Should it perform better than the current search function? Should it answer every question, even if the answer is not in the knowledge base? Or is it enough for it to provide good answers only to questions for which you know the answer is available?
“First define what ‘good’ means,” explained Ernout Douqué, CTO of Conclusion Intelligence. “Ask subject-matter experts to determine the correct answer to a series of questions. This becomes the benchmark against which you test the application. The technology cannot do that for you, because this knowledge resides with the people in your organisation.”
Sometimes, this also calls for a new role. You need someone who understands what the business requires and can translate that into something measurable.
Once you have defined what ‘good enough’ means, you can work faster and more safely. You can immediately see whether a change improves the application. You can also demonstrate why an application is ready to go live, for example through documented test results. Embedding this approach also captures the knowledge held by your own people. Because many companies use the same AI models, this knowledge will distinguish your application from a competitor’s.
Bottleneck 3: No owner, limited adoption
If an application is good enough, we can say that it works. But whether it delivers value for the organisation depends on who is responsible for it and whether people actually use it.
Start with accountability. As soon as an AI application is put into use, ownership must be assigned. “Responsibility often ends up with IT or the data team. They can build or manage an application, but they are not responsible for the working process it forms part of,” said David. “It is therefore better to assign ownership to the colleague or team responsible for the outcome of the work itself. They understand what the application means for customers or for financial performance. Only then will AI deliver lasting value.”
Then there is adoption itself. “Many organisations provide a tool, offer training and assume that people will naturally start using it,” said Mirte van Drunen, Business Consultant at Conclusion Consulting, during the session on human change and adoption. “But employees follow their own journey. They first need to understand why the organisation wants to use AI and what that means for their work. Next, they need to be willing to use the tool; only then does the e-learning module become useful.”
AI enthusiasts have often already started experimenting themselves, or may even be AI champions. They will begin using the application straight away. More cautious colleagues need to be involved earlier in the process. A single approach to adoption will therefore not work for everyone. Different colleagues move at different speeds and have different questions. How you measure usage also varies by group. For a knowledge worker, you might look at how often they use the tool; for a contact centre employee, whether they trust the output and use it in their conversations.
There is no point waiting until everyone fully trusts the AI tool. Trust grows through experience. Give people room to learn and experiment within clear boundaries, and let enthusiastic early adopters bring their colleagues along.
Bottleneck 4: Starting from scratch every time
One successful AI use case creates an appetite for more. But if every team has to work out security, legal and data requirements from scratch, the process returns to square one with each new application. “Set up the essentials every application needs properly once, such as who has access, how quality is checked and what it costs,” said Ernout. “The first application requires the most work, but each subsequent one becomes faster.” This was the guiding principle during the session on platforms, technology and data: use case number ten should be faster, safer and less expensive than the first.
Not every application needs to be assessed in the same way. “The greater the risk, the more rigorous the assessment,” said Skip. “An application that affects customers, influences decisions or uses sensitive data receives a full review and has a named owner.” Other applications can follow a simpler route. Using building blocks from previous successful use cases, a team might assemble an internal tool themselves, for example. This leaves room for colleagues who want to experiment. If such an experiment proves successful, it can in turn be made available as a building block for other teams. “In this way, the foundation grows in step with what teams discover in practice,” said Skip.
These days, a pilot for a new AI application can be up and running quickly. During AI on Stage, Marcel Antons, Director Strategy & Innovation at myBrand Conclusion, compared AI to a confetti cannon that has got out of hand: “fantastic, but when you look at the floor afterwards, the result is not always quite so attractive.”
Structure is essential, but it must not come at the expense of creativity. You build that structure through the decisions surrounding the technology. What should an application deliver, when is it good enough, who owns it and how do you bring colleagues along? These factors determine whether a pilot succeeds or ends up gathering dust in a drawer.
