No organisation would roll out a new ERP or CRM system without proper training, an adoption plan, or a way of measuring users’ skills and knowledge. It is widely accepted that the success of a technology implementation depends on whether people understand the system they are expected to use. Strangely enough, we tend to abandon that principle when it comes to AI. Its use is growing organically across organisations, while investments in tools and licences continue to increase. But how much attention is paid to the knowledge level of the people actually working with AI systems?
Research shows that AI literacy and understanding of AI terminology among the Dutch workforce scores just 5.1 out of 10. At the same time, most people believe they understand the basics. In other words, knowledge is lacking, but confidence is not. That combination makes AI implementations particularly vulnerable.
A blind spot beneath the bonnet
What we experience ourselves tends to stay with us. The same applies to AI. Most people know what a prompt or a deepfake is because they encounter these concepts in everyday life. But when it comes to understanding how a large language model (LLM) works, or the neural networks that underpin it, knowledge quickly runs out. That understanding does not develop naturally through day-to-day use. It requires deliberate effort and curiosity. Not everyone is willing to invest that effort.
That is true for people in your organisation as well. Daily use creates the impression that they understand what is happening behind the scenes. One telling misconception is that almost half of the Dutch population (47%) believes generative AI is essentially a sophisticated search engine that summarises existing information. Only a minority understands what it actually does: generate new content such as text, images, audio or code based on patterns learned from vast amounts of data. The result is that AI adoption is outpacing AI literacy within organisations.
Misunderstanding the technology leads to misusing it
At organisational level, this knowledge gap becomes a usage problem. The way people think AI works shapes the way they use it.
If you assume AI searches through existing sources, you are likely to approach it as though it were a database and expect its answers to be factually correct. If, however, you understand that AI generates responses based on probabilities and learned patterns, you engage with it differently. You stop treating its output as established fact and start evaluating it more critically.
As a result, the same tool can lead to very different behaviours depending on the user’s understanding.
The real challenge is that misconceptions do not automatically disappear through repeated use. AI continues to provide convincing answers even when it is treated as a database. In fact, its apparent confidence often reinforces the misunderstanding. Yet once those answers relate to your organisation, customers or areas of expertise, that becomes a genuine risk.
A general-purpose language model has no inherent knowledge of your organisation’s specific data and may confidently produce inaccurate information. Raising awareness among employees is therefore an essential first step in addressing misconceptions about AI. The associated risks around company information can also be mitigated technically, for example through Retrieval-Augmented Generation (RAG), which enables an LLM to draw on trusted internal data sources to improve the relevance and reliability of its output.
Delivered with confidence
Even when employees use AI appropriately at the front end, what happens next is just as important. The effectiveness of AI depends not only on how prompts are written, but also on how output is evaluated.
By now, “always verify AI-generated content” has become something of a mantra. Most people understand this in theory. In practice, however, that is very different from actively questioning an answer in the moment. AI often comes across as a confident colleague, which makes people less inclined to doubt what it says. Yet that scepticism is exactly what is required.
The certainty with which AI presents information is not a sign of accuracy. It is a design characteristic. AI systems are built to communicate with authority, regardless of the question being asked.
Of course, not every response requires exhaustive verification. But if users do not recognise that AI can hallucinate, they are unlikely to judge correctly when verification is necessary. As a result, unverified output quietly finds its way into everyday work.
The same confidence can also blur accountability. When something goes wrong, responsibility does not sit with the AI tool, its developer or the IT department. It remains with the person using it. Once that reality is fully understood, people become more critical of AI-generated output and more deliberate in how they apply it to their work.
From thinking you know to actually knowing
If you deploy AI in an organisation where people believe they understand it, but in reality do not, you are building on shaky foundations.
Effective AI implementation begins with a realistic assessment of users’ knowledge levels, just as it would for any other technology rollout. Measure AI readiness by AI literacy, and make a knowledge assessment a standard part of every AI programme.
An organisation is only truly AI-ready when its people understand not just what the technology can do, but also, and perhaps more importantly, what it cannot.
