如何同時衡量模型的正確性與長期穩定性？

 

如何同時衡量模型的正確性與長期穩定性？

Home Credit - Credit Risk Model Stability

Create a model measured against feature stability over time

https://www.kaggle.com/competitions/home-credit-credit-risk-model-stability/overview

gini = 2 * AUC - 1   (AUC:0~1)

stability metric = mean(gini) + 88.0 * min(0, a) - 0.5 * std(residuals)

針對預測能力呈現下滑趨勢給予懲罰 :即 a < 0 的情況

穏定性評估: 計算 residuals 的標準差, 愈小表示模型愈穏定. 最好是 0

[References ]

Understanding ROC and AUC