Predictive Model Training & Backtesting
Models follow train/validate/test splits with walk-forward validation. Backtesting against unseen data prevents overfitting.
NEI Predict builds forecasting models using historical data. Every model goes through a rigorous train/validate/test cycle before deployment.
Data splitting: 70% training, 15% validation (hyperparameter tuning), 15% holdout test (final evaluation). The holdout set is NEVER used during training. If a model performs well on training but poorly on holdout, it is overfit and rejected.
Backtesting runs the trained model against historical periods it has never seen. The system measures accuracy, precision, recall, Sharpe ratio (financial), and maximum drawdown. A model below baseline (buy-and-hold for financial, naive forecast for demand) is not deployed.
Walk-forward validation: train on expanding windows, test on next period iteratively. This reveals how models adapt or fail during regime changes, market shifts, and structural breaks.