This summer we celebrated our 4th anniversary at Enjins. In addition to a few well-deserved bottles of champagne and a great office party, we also reflected on previous machine learning implementation projects in our scale-up portfolio. Though I can come up with more than 100 lessons learned (and maybe will do so later), it seems more fun to pick a few quotes that resonate with us. To me, these 4 quotes, on ML & AI implementation, perfectly capture the challenges we typically see in developing and implementing ML and AI systems that drive business value. I believe, especially for the data and ML experts out there, those are recognizable. Please add, disagree, or comment!

“The first question you should ask yourself when building ML is; can we do it without ML?”

or why AI is not the solution to everything.

We often use Machine learning to partially automate repetitive decisions. For instance, when deciding if claim “A” is valid or not, whether we should focus on customer “X” and if it is a case of fraud, etc. However, a machine learning algorithm is not always the answer to the problem. Even when you have taken care of the technical challenges successfully, ML remains an additional layer of complexity for organizations. Complexity, such as the involvement and change management of business experts, the necessary explanation of model output and performance, and the frequent updates of models. Since complexity can be an adverse circumstance for a company’s scalability, we put a premium on simplicity and acceleration. Also, at the cost of your model performance.

💡 Learning

Ensure to carry out checks early on where ML can and can’t help. And see how much additional complexity is required to make success of your use case. This depends heavily on an organization’s data & AI maturity. Our experience shows that a decision support system with business rule logic can be wise as a first step and creates a foundation that allows for more complex (ML-based) methods over time.

“Our wizards are finally there!”

or how to manage AI expectations.

This quote might be familiar to data scientists, especially for the first data science hire within a department or organization. Similarly, we hear this when being the third party working on the first AI or data science project. Expectations are unrealistic and fuzzy at the beginning. Impatience and frustration can grow when it’s unclear why the implementation takes time (like with every engineering project). Management of expectations will always be part of building AI solutions and demands our awareness steadily. What might be very logical for an ML engineer in terms of what is possible and what is not, can be easily overseen or marginalised by business experts. It is crucial to be aware of this; it is part of the challenge and part of your role.

💡 Learning

Create awareness of the implementation process early on to manage (high) expectations that come with AI; mention timelines, potential bottlenecks, and needed involvement to overcome them.

‘“We will just be able to do more, with a little help from our robot friends! “

Or how you should not underestimate change management in AI projects.

As the first quote explains, ML is often applied to partially automate a repetitive decision. The goal is to create an algorithm-human interaction where the algorithm does the bulk and the human does the difficult edge cases. Such interaction requires trust from the expert in the algorithm and the other way around. Typically, these experts have worked for multiple years at the company, and have been doing their job with a great amount of experience and knowledge. They are typically proud of their knowledge and work, and rightfully so. Therefore, change is not straightforward and should not be underestimated. We have noticed this challenge even at younger scale-ups, where people are open to change. To overcome this challenge, we like to adopt the mindset that we work for those that use our AI product. Not for those that are actually paying for the project.

💡 Learning

By involving the end users from day one, you can navigate through the difficult change process that comes along with ML models. In many applications, the algorithms take over decisions that were previously made by experts. Gaining their trust will both improve your solution/model as well as make it possible to go to a successful implementation.

“A data scientist’s laptop is the most famous graveyard for data science models.”

or how you should always think of MVP rather than proof of concept.

This quote is one of the reasons why we founded Enjins. A data science model should not die on a local laptop; it should, in the end, be implemented into your tech, organization, and process such that it creates business value. Throughout the last four years, we have seen a definite shift where more and more companies successfully create sustainable business value with AI. On the other end of the spectrum, we also see companies with more than 50 data science initiatives, out of which 1 or 2 (!) are actually implemented and create day-to-day business value. An important underlying part is that data science experts don’t originate from the software engineering space, but rather from the statistical and mathematical space. They are good at modeling but lack programming skills in a complex tech stack. The ML engineers’ role effectively fills this gap and assists the data science team beyond modeling.

💡 Learning

Start thinking in terms of Minimum Viable Products (what is needed to get the first version of this use case live and actually create some first business value?) rather than proof of concept (can we make a model on historical data that is accurate enough?). If the focus only goes to an offline proof of concept, requirements like latency (is the solution fast enough?), data availability (can we actually make the prediction in a live setting), process requirements (do company experts and other users feel confident in using the model’s outcome or suggestion?) and an integration roadmap (what would be a good subgroup or country to go live?) are frequently overlooked.

Agree or disagree with the 4 quotes on ML and AI implementation? Let me know in the comments!