Multicalibration

Multicalibration is a statistical fairness property that asks a prediction system not only to be accurate on average but also to give trustworthy probability estimates for every group defined by sensitive attributes such as gender, race, age, or any combination of those. In practical terms, if the system says there is a twenty‑five percent chance of an event for a particular subgroup, then roughly one quarter of the individuals in that subgroup should experience the event over the long run. The requirement is applied simultaneously to many overlapping groups so that no intersection can hide systematic bias.

The importance of multicalibration lies in its ability to prevent hidden disparities that can arise when a model appears well‑calibrated overall but misestimates risk for minorities or other protected classes. By guaranteeing accurate probabilities across all specified slices of the population, decision makers—whether allocating medical resources, setting insurance premiums, or approving loans—can rely on the same confidence level regardless of who is being evaluated. This reduces the chance that disadvantaged groups receive systematically worse outcomes due to mis‑calibrated risk scores.

Multicalibration shows up in settings where predictive models inform high‑stakes decisions and where fairness scrutiny extends beyond a single protected attribute. It is used in algorithmic credit scoring, criminal justice risk assessments, personalized medicine, and any domain that wants to audit its models for subgroup reliability. The concept also guides the design of training procedures that adjust model predictions until the multicalibration condition is satisfied across a pre‑chosen collection of groups.