E-Commerce Conversion — Worth Shipping?
A purchase-intent model validated forward in time, then benchmarked against a one-line rule.
Problem
It's easy to report a great-looking conversion-prediction score and much harder to know whether the model is worth deploying. Random train/test splits and near-leakage features flatter models that then fall apart in production.
Solution
On real e-commerce session data, this project catches a near-leakage feature, re-validates the model forward in time (train on past months, test on unseen future months), and benchmarks it against a trivial business rule — then honestly recommends against shipping the model when the rule wins. The rigor is the deliverable, not the score.
Architecture
UCI Online Shoppers data flows through a preprocessing pipeline with a pandera data contract enforced before every prediction. Logistic-regression and XGBoost models are compared with and without the leaky feature, under both random and temporal splits, with feature-drift checks (PSI/KS) and isotonic calibration. MLflow tracks experiments and registers the model; a FastAPI service serves calibrated predictions. SHAP explanations, a model card, and a challenges log document the reasoning.