The staggering rate at which data is generated often outpaces an organization’s ability to use it, resulting in an abundance of dark data. According to a recent CDOTrends article, as much as 55 percent of a company’s data is considered dark.
As we’ve written about previously here at the APEX of Innovation, some risks emerge as companies grapple with dark data. But with the right approach, organizations can mine dark data and transform it into real business opportunities. For example:
Coded Audio from Race Car Drivers
In a recent CIO piece, Maria Korolov outlines how Envision Racing, a UK-based racing team, combined audio recordings from drivers and car sensor data to improve its racing strategy. Using natural language processing, the organization built deep learning models that produced AI-driven predictions and insights in a matter of seconds—far faster than humans can. As a result of tapping into its dark data value, the Envision Racing team took first and third place at the ABB FIA Formula E World Championship in July.
This example highlights the benefits of dark data generated by humans and intended for consumption by other humans, not machines. While it was audio recordings in the Envision Racing case, numerous other examples can be beneficial in an enterprise setting—emails, chats, and other forms of communication. Because data warehouses aren’t typically set up for these data types, the best approach is to leave the information where it currently resides and adds a layer of indexing and metadata for searchability.
Uncovering New Revenue Streams
Dark data also has the potential to create new revenue streams and support more efficient processes that could ultimately reduce costs and increase competitive advantage. Think about dark data such as unsolicited feedback—if properly analyzed, this can provide insights into the customer experience, help understand machine operating capacities, or even gauge employee morale and productivity.
Effective Governance is Essential
Often companies are hesitant about leveraging dark data because they lack confidence in the information. Establishing effective governance can help overcome these concerns, as observability and traceability engender greater confidence in the underlying data and subsequent AI models.
To address these governance considerations and other examples of dark data use cases, look at Korolov’s CIO piece.