Model Monitoring & Drift Detection
Model monitoring detects performance decay, data drift, and concept drift. Retraining only promotes models that outperform current live version.
Deployed models degrade as conditions shift. NEI Predict continuously monitors performance and detects drift before it causes losses.
Performance monitoring tracks live predictions vs actual outcomes daily. If rolling 30-day accuracy drops below deployment threshold, retraining alert triggers. Models have expiration dates.
Data drift detection compares incoming feature distributions against training distributions using PSI (Population Stability Index). PSI exceeding 0.2 for any key feature signals meaningful shift.
Concept drift: feature-target relationships change even when distributions stay stable. Detected via residual analysis. If prediction errors show systematic patterns (consistently wrong in one direction), concept drift is flagged.
Automated retraining pipelines retrain on latest data, re-validate, and promote new model ONLY if it outperforms current live model. A retrained model that performs worse is rejected.