The logistic model is the only one that detected held-out closures in the reported folds, but its precision is low and its recall uncertainty is very wide. Gradient boosting has a higher ranking AUC, yet it detected none at the selected threshold. These measures answer different questions and do not establish a calibrated closure probability.
The real question isn't "which model has the best F1?" It's "which model would you trust a policy decision to?" A simple rule that anyone can audit beats a complex model that nobody can explain — especially when the stakes involve accreditation and financial aid decisions that affect hundreds of thousands of students.
Side by Side
| Model | Recall | Precision | F1 | AUC (area under curve) |
|---|---|---|---|---|
| Stress Score (3 rules) | 0% | 0% | 0.0 | — |
| Logistic Regression | 40% | 1.6% | 0.031 | 0.849 |
| Gradient Boosting | 0% | 0% | 0.0 | 0.901 |
AUC measures ranking across thresholds; recall and precision describe the selected operating threshold. The higher gradient-boosting AUC therefore does not contradict its zero reported recall. With roughly one positive case per validation fold, none of these estimates supports an operational choice.
What the Model Learned
The gradient-boosting feature ranking is led by 10-year earnings (42.3%), followed by enrollment (15.8%) and retention rate (10.4%).
Feature importance describes how this fitted model split the available data; it does not show that a feature causes closure or will generalize to another cohort. With six positive cases, these rankings are best treated as leads for future validation.
Do not deploy any of these models from this result. The stress score is the easiest to audit, but it missed the observed closures at its selected threshold. The ML models require more outcome data and external validation.
5-fold stratified cross-validation on 1,516 institutions with 6 observed closures. Metrics are exploratory and are not calibrated institution-level forecasts.