Good Models (on their own) don’t generate sufficient value
How to realize business value from highly performant models, using a case study in the field service industry.
The field service industry (like other industries) moves toward automated tasks or augmented existing processes based on AI tools that provide decision support to field technicians. Using machine learning is the key to future success but doing it right is vital to getting the most out of your investment.
Companies that develop machine learning models assume that if the models provide excellent predictions, all the hard work was done, and now all that is left is the easy part of assimilating the models into the work processes — This assumption is wrong.
No matter how good the model’s performance is, at the end of each model stands a user who needs to use it, and if the business process flow does not fit well for the user, the model will lose its true meaning.
Machine learning models must be incorporated with business process flows, and without diving into the depths of that workflow, it is impossible to succeed.
Before applying a machine learning model to a business process, we first need to ask some questions:
- What part of the business process are we augmenting or automating?
- How can we measure its performance?
- And essentially, How do we expect this use of AI to improve the process?
In many cases, defining a task will mean breaking down some larger problems into smaller ones.
How to utilize a model to reduce multiple visits?
In this case study, the application is using machine learning to predict the chance of repeat visits, we will have to answer questions like:
- What quantifies the quality of predictions?
- How should the system and service technicians respond to the model’s prediction — multiple visits vs. single visits?
- How do we measure the extent of success — is it only by a reduction in repeat visits?
Hence, we need to look at it more closely and ensure that every part of the process flow integrating the machine-learning solution has been considered clearly before beginning implementation.
Back to the case study, if the model predicts that a particular service order will likely have a repeat visit, the system process flow should recommend ways to prevent it. Only displaying the probability value in the application is not enough. To do this, we need to understand the reasons for multiple visits.
What can lead to a repeat visit?
A study from the Aberdeen Group broke down the causes of low FTFRs (First Time Fix Rates) as follows:
92% of the time, it seems, the cause is poor communication — either the appropriately qualifeid engineer has not been sent, the correct information has not been given to them, or they weren’t able to gain access to the site in the first place.
Therefore, to improve the process flow, emphasis must be put on improving communication.
Predict repeat visits workflow
The below figure illustrates the revised flow that starts when a service order with a high chance of repeat visit is assigned to a technician.
Conclusion
AI and Machine learning has become an essential tool for companies to automate processes, and many companies seek to adopt algorithms widely. It is crucial to conceive entire business processes that involve machine learning models from beginning to end.
Think end to end — It is required to break apart how work should be done and design the process more conducive to how machine learning models and people operate together.
Following our case study, a significant challenge in the field service industry is sending technicians to job sites with the right information and equipment to complete the job on the first visit. If the system displays only the probability of a return visit to the technician, he will most likely ignore it, and the model will lose its meaning. Therefore, we need to design a process that increases technician awareness. For example, a workflow behaves like a checklist that provides recommendations from similar past incidents or highlights a relevant snippet from technical guidelines.
The key to success is providing the technicians with up-to-date information and customer service history at the right time so they can arrive at the customer site ready to complete the job.
Developing a machine learning model can leave value on the table when there is not a suitable workflow realizing the value.
