[Google deepmind-github] Evaluation and calibration with uncertain ground truth

This repository contains code for our papers on calibrating [1] and evaluating [2] AI models with uncertain and ambiguous ground truth.

​https://github.com/google-deepmind/uncertain_ground_truth

​[1] Stutz, D., Roy, A.G., Matejovicova, T., Strachan, P., Cemgil, A.T.,

    & Doucet, A. (2023).

    [Conformal prediction under ambiguous ground truth](https://openreview.net/forum?id=CAd6V2qXxc).

    TMLR.

[2] Stutz, D., Cemgil, A.T., Roy, A.G., Matejovicova, T., Barsbey, M., Strachan,

   P., Schaekermann, M., Freyberg, J.V., Rikhye, R.V., Freeman, B., Matos, J.P.,

   Telang, U., Webster, D.R., Liu, Y., Corrado, G.S., Matias, Y., Kohli, P.,

   Liu, Y., Doucet, A., & Karthikesalingam, A. (2023).

   [Evaluating AI systems under uncertain ground truth: a case study in dermatology](https://arxiv.org/abs/2307.02191).

   ArXiv, abs/2307.02191.

​