Concept Drift

What is Concept Drift?

ML models have the primary objective of extracting patterns from historical data and predicting future behavior based on these past patterns. More technically, these patterns are nothing but concepts that represent the relationships between variables.

When these relations change in the real world, it also affects the predictive capability of the ML model. The real-world changes in variables result in invalid patterns learned by ML models. This phenomenon is termed “concept drift.”

For example, a personalized recommender system profiles user shopping patterns based on past behavior. But significant changes such as customer relocation to a different geography or a global pandemic like COVID-19 drastically affect the shopping behavior making current recommender systems irrelevant.  

Concept Drift Classification

Sudden drift

Drift occurred as a result of drastic external changes. It can be a sudden/abrupt change such as mobility patterns affected by the COVID-19 outbreak.

Incremental drift

Drift occurred due to small changes and noticed only after a long period. For example, changes in the loan default pattern over some months requires retraining of a credit scoring model.

Recurrent drift

This drift happens periodically, maybe during a specific event or time of the year. For example, the Black Friday event significantly affects consumer buying patterns. Therefore, training a different model on Black Friday data makes sense.

‍

Why is Detecting Concept Drift Important?

Due to concept drift, model predictions might become less accurate over time, or opportunities to scale up the accuracy might be missed. Hence ML models need to accommodate the variable and target changes quickly and accurately.

‍

Addressing the Concept Drift Challenge

Dealing with concept drift challenge is a two-step mechanism, with the first step being concept drift detection and the second step being addressing the drift. This is usually done using periodic retraining, online learning, or feature dropping.

However, detecting concept drift in models running in production is highly challenging. You can conduct a manual check of the models periodically to ensure consistency in your models. But a much better solution would be using a monitoring tool.

An ML monitoring tool like the Censius AI Observability Platform monitors ML pipelines, detects drifts, and alerts users before these minute drifts add negative business value. Its concept drift monitor uses the ‘Early Drift Detection Method’ to monitor the frequency of output classes and notice significant variations. 

Such an observability platform will help you automate and accelerate your ML projects so that your ML models run smoothly in production.

Further Reading

Machine Learning Concept Drift – What is it and Five Steps to Deal With it

Best Practices for Dealing With Concept Drift

‍