The Business: Since 2011, Yource (also known as Vlucht-Vertraagd.nl) has been helping airline passengers successfully file claims for compensation for delayed, cancelled and overbooked flights. Every single day, up to a 1,000 new claims are filed at their website.
The Challenge as described by Mario Wester (CTO Of Yource): “In the past eight years we have expanded significantly, resulting in a rich and ever-growing dataset containing both claims and flight information. This information was used by our claim experts in all steps of the process, but this was a time consuming task. Our ambition was to speed up and partially automate the claim process. This is why we started cooperating with a Machine Learning company like Enjins.”
Mario continuos about the first step in the process, the validation of a claim:
“Our team at Yource consists of over 100 experts handling the claim process. The first step in this process is claim validation; the expert does a first check on whether the customer’s claim is entitled for compensation. An expert needs 3-7 minutes for this process. Considering the growing amount of claims, experts spend most of their time on this repetitive and relatively easy step in the process. Therefore, only limited time is left for the more complex tasks further in the claim process. Furthermore, the backlog grows, blocking our further growth. To realize further scalable growth of the company, we need to automate the validation step in the process.”
The main question in this step was whether we can predict if a certain claim is going to be successful or not. When experts make this decision manual, they use several internal and external information. To get all these data sources together and store them in one place, a scalable and future-proof data infrastructure is needed. The overall goal is to get Machine Learning in production within the existing technology stack at Yource.
The Solution: Learning from the experts, we got to understand the whole process and existing technology at Yource. We worked together closely with the business and development teams to build production-ready ML engines to automate the claim processes. Every claim that comes in to Yource is now first sent to the algorithm before it is sent to an expert. Only edge cases are handled manually.
Mario: “The claim validation that used to take an expert around 5 minutes is now handled in less than one second for more than 70% of the claims. This saves our experts lots of time, making them available again for the more complex tasks.”
Next to creating the ML models, Enjins created a data infrastructure to train, test and run models live in production. The data infrastructure is realised in such a way that models for new steps of the claim process can be deployed in a similar fashion.
Before machine learning took over parts of the manual work, it was crucial to run pilots and take feedback from experts into account. We took several steps to work towards production:
- Starting with a silent pilot, in which employee did not see the live predictions in the background.
- Developing a constant feedback loop. Employees can overwrite the predictions of the model and this way give valuable feedback on the performance of the model. Leading to for example adding new external data sources to feed relevant extra information to the model. By using this feedback loop, constant improvement of the Machine Learning Engine is possible.
- Finally, after extensive testing, the ML models run in production and take automated decisions based on a threshold strategy.