How do we implement machine learning ops within our data tech stack?

Avoiding lock-in 

When it comes to technology and new tools, it is easy to be swept up in the excitement and hype cycle of new developments, but there must be considerations about which platform(s) should be selected. For example, some tools are complex and require too many skills to be used effectively across an organisation.   

“A challenge with selecting a commercial platform is that there is a risk of getting locked-in to vendors,” explained Simon. “Often, there are no easy ways to transfer models across platforms and the needs of the business will invariably evolve which risks outgrowing or moving away from the specialisms of the selected tool. Participants of the roundtable noted that the use of auto ML capabilities meant the lock-in risk was even higher.” 

Sourcing the data needed 

Part of the issue implementing ML ops into a tech stack is being able to find and utilise the data sets required for success. This was a common hurdle faced by members of the roundtable as this task would usually be required at the start of a project when there was minimal capacity and an eagerness to prove proof of concept to decision makers.  

There were calls from the roundtable to create data catalogues and dictionaries for internal use, as well as to improve the storytelling and data literacy capabilities of the team. Some felt they knew already where their data was located, with one business representative explaining how their organisation had been actively migrating from an on-premises system to a cloud platform. This transformation had taken over two years and was difficult, but because the data team were so involved in the process they know exactly where the data sets they require are and the lineage of the data. 

A smaller, but by no mean insignificant hurdle faced by the group was that of team member churn. When an established member of the team left, there would often be a large knowledge gap in their absence that made tasks such as collating the right data slower. An issue found with higher churn is that the response times to problems are slower and the ability to fix the problems before they scale is reduced, and this is arguably something that ML will not be able to address without the human skills behind it.