The Data Wars

Talking to a BI manager the other day I saw that there might be a war brewing in many companies – well, maybe not a war but a competition. It’s the competition between various analytical information suppliers.

Centralized IT departments have had a long standing monopoly on implementing and operating the systems necessary for delivering high quality analytical information for decision makers. They still do. However, technological advances in integration technology, data analytics and data visualization is eroding the platform on which this monopoly is based.

IT departments are more often than not internal cost centers charging for their services. When managers need to gather data, clean it, merge it, process it and present it, it will cost them money. The IT department will guarantee that the data is as valid, reliable, accurate and complete as humanly possible. But this comes with a price tag that some managers are unwilling to pay. It also takes time to ensure that the data is of a quality that conforms to the aims and standards of the IT department.

Today, other departments are acquiring the means to process and analyze data as well. Accounting departments, risk analysis departments, internal control departments, logistics departments, and planning departments are all engaged in gathering data, processing it and presenting it for decision makers. These departments have access to databases, can integrate data to some extent and prepare it for presentation and analysis.

These departments are capable of servicing their own managers with ad hoc “good enough” data gathering and analysis at a much lower cost than the IT department will charge them. They are also capable to supply other managers in the organization with analytical services at a lower cost than the IT department. This data and analysis might not based on the data quality processes performed by the IT department, might use departmental tools (read: Excel) and based on the access levels to databases this department has. But to many managers the result might seem “good enough” and give them the information they need.

Is this a problem? Or is it a natural step in the evolution of the “democratization” of data and end-user empowerment? I think it at least merits a thought regarding the future structuring of IT departments and the services they offer. Top heavy IT departments with cost performance requirements, fixed cost markups, and rigorous complex administrative procedures seem out of sync with the development of the analytically savvy end user, user friendly integration and analytical technologies and increased access to data.