A recent InfoWorld article examined a major barrier that organizations often face when attempting to scale data science initiatives: data science toil.
The article describes data science toil as the combination of inefficiencies, custom workarounds, tribal communication, non-interchangeable MLOps flows, redundant efforts, and shadow IT that typically appear as companies expand beyond a limited number of models in production. According to the piece, data science toil is inevitable when data science is scaled organically rather than purposefully and increases risk, cost, and resource demands at the expense of more strategic activities.
So, what should you do to avoid these and other negative consequences? Read on for three common causes of data science toil and how to address them in your organization.
- Lack of Ownership from IT Leadership
Despite widespread awareness about the importance of data-driven decision-making, data science is still too often treated as a technical discipline rather than the enterprise capability it actually is. When this happens, it’s all too easy for security, reproducibility, and governance issues to arise, which, in turn, leads to redundant work, productivity issues, and inaccurate insights across different business groups. In addition, lack of IT leadership involvement effectively opens the doors to shadow IT. To get ahead of these problems and eliminate the associated data science toil, CIOs must step to the plate and take charge of data science leadership.
- Lack of Standardized MLOps Processes
Another task for IT leaders trying to avoid data science toil is to bring standardized flexibility to MLOps. Ensuring that there is a clear, intuitive best practice for functions like data pipelines, research, paths to production, and asset maintenance will enable self-service for data scientists and their research in an IT-approved manner.
- Lack of Shared, Collaborative Research
Knowledge management in data science is a pervasive problem, but one that is not necessarily well-recognized by IT leadership. Because this is a common driver of data science toil, it’s important that IT leaders integrate strong project management capabilities with data science tools. This will help data science teams collaborate more effectively and increase knowledge sharing between teams.
As organizations look to scale their data science investments heading into the new year, IT leaders must take a more proactive role in addressing data science toil. For more on how to do this effectively, check out the InfoWorld article in its entirety.