AMIS Conclusion research reveals data engineering challenges and untapped data potential.

42 per cent of large companies do not always gain the insights they need because they have too much data. Only 39 per cent believe that data’s contribution to business objectives outweighs its costs. These are findings from research commissioned by AMIS Conclusion among digital leaders at companies with at least 250 full-time equivalent employees.

Almost drowning in oceans of data

Fuelled partly by the big data hype, companies have collected oceans of data. In many cases, however, big data has become fat data. Excess data becomes an unhealthy burden, with all the associated drawbacks and risks. For example, 40 per cent of the 150 respondents say the origin and quality of their data are not entirely clear. Another 31 per cent say data is not available quickly enough when needed, while 27 per cent report that they cannot always access the data they need.

Data hoarding: risky and often pointless

Many companies hoard data, collecting large volumes without a clearly defined purpose. For example, 63 per cent deliberately retain more data than they currently use for potential future use. This is a risky and often pointless strategy: the data is frequently too incomplete or insufficiently detailed to be usable later.

Conclusion: companies struggle with data engineering

The research findings are clear, says André van Dalen, Managing Director of AMIS Conclusion: “Although most companies say data-driven working is a priority, in practice they face numerous challenges in data engineering, real-time data and data platforms. As a result, they miss out on insights, cannot make real-time adjustments, and their hired data scientists and machine learning specialists cannot work efficiently. This reduces their effectiveness and flexibility.”

A call to action for the IT sector

Van Dalen says: “The IT sector first encouraged customers to collect vast amounts of data, then left them to work out how to do something meaningful with it. It is time for us as a sector to acknowledge our responsibility and help customers develop data products that deliver real value.”

AMIS Conclusion is therefore calling on the sector to act. Van Dalen explains: “Help customers make their data landscapes more manageable and professional through high-quality data platforms and data engineering. From now on, our first question to customers will be: is your data as high-quality and reliable as the water from your tap?”