How to Take the Next Step: From MVP to Full-Scale Product

Data Analytics and Big Data already brought some changes to the infrastructure landscape with them, like massive storage capabilities for different kinds of data as well as distributed systems for processing. AI and in particular ML has additional needs when it comes to performant computing, like GPUs or TPUs that need to be easily accessible to developers. So before getting started, make sure you have the necessary infrastructure (in the cloud or on-premises) to bring the power of AI to your business application. 

New tools that support some parts of the ML lifecycle pop up almost every day. Be aware that evaluating them by functionality and potential for adoption requires extensive scanning of the field and deep technical knowledge for comparisons of differences and overlaps in capabilities. There is no need to always stay up to date with the most cutting and therefore sometimes bleeding edge tools, but keep in mind that you’ll need to establish a system and processes to stay informed and be able to introduce relevant new tools regularly. 

Given the vast ML(Ops) Tool Landscape, it is tough to choose the right tools. They should fit together nicely and fulfill the requirements of individual use case projects or even across multiple use cases within a department or organization. Depending on the respective maturity level, covering some requirements with a selection process is more important than with others. 

IT Governance does not stop at ML solutions. Setting access controls for all models in production, versioning all models, creating the right accompanying documentation, and monitoring models and their results is paramount to adhering to existing IT policies. Responsibilities for this new type of artifact need to be clarified and underpinned with outlining the requirements as well as the organizational authorities.

As the stakes and complexity around AI/ML initiatives grow, so does the need for collaborative, cross-functional AI/ML teams that span departments. Strong, sustained results absolutely depend on close alignment between multiple departments and stakeholders. 

Based on the recent report from Algorithmia, only 11% of organizations can put a model into production within a week, and 64% take a month or longer. If you are rather new to the ML game, your organization probably will belong to the latter group. Since up until deployment the new model does not create any value, the training to deployment time is a crucial KPI to evaluate ML maturity of an organization as well as individual projects.​ 

38% of organizations spend more than 50% of their Data Scientists’ time on deployment —and that only gets worse with scale. Data Scientists usually focus on algorithms and models to solve business problems but are not as well versed in software development and in particular infrastructure topics. Let your Data Scientists focus on what they are best at and have ML engineers support them with topics like the deployment of ML models.  

Be aware that the way you deploy ML models and in which fashion they are served influences whether the use case requirements can be fulfilled. So, make sure to consider all different aspects to select a suitable deployment pattern. There is e.g. the option to do predictions in a batch or one-by-one on an as-needed basis. You should also consider that models can be provided as part of the user application or as an endpoint. Also, the deployment as well as the serving setup changes in case of high availability needs, e.g. in use cases involving data streams.  

ML systems need extensive testing in additional areas compared to traditional software applications. In particular, tests regarding data and the ML model increase complexity and scope within the classical test pyramid. 

Within ML applications there are multiple areas with potential for automation. Since software development best practices are often not yet fully adopted, CI/CD pipelines are one area of automation often missed. With some adaptions, they can however bring the same value to AI projects as to classical software projects. Even more so, due to the increased amount of (automated) testing needed for a fully fletched ML system.