probcal

Calibration of Binary and Multiclass Probabilities

Provides S3 calibrators, metrics, and diagnostics for binary and multiclass probability calibration. Binary methods include Platt scaling, temperature scaling, beta calibration, histogram binning, and isotonic regression. Multiclass methods include temperature scaling, vector scaling, Dirichlet calibration, and a one-vs-rest wrapper for the binary calibrators. A calibration-inference layer adds debiased calibration errors, bootstrap confidence intervals, and a kernel calibration hypothesis test for binary and multiclass predictions, including the strong (canonical) multiclass case. Methods follow Platt (1999, ISBN:9780262194488), Zadrozny and Elkan (2002) <doi:10.1145/775047.775151>, Guo et al. (2017) <https://proceedings.mlr.press/v70/guo17a.html>, Kull et al. (2017) <doi:10.1214/17-EJS1338SI>, Kull et al. (2019) <doi:10.48550/arXiv.1910.12656>, Widmann et al. (2019) <doi:10.48550/arXiv.1910.11385>, and Kumar et al. (2019) <doi:10.48550/arXiv.1909.10155>.

Versions across snapshots

Dependencies (latest)