Biography

Deputy Head of the Department of Intelligent Medical Systems at the German Cancer Research Center, where she leads the Validation of Intelligent Systems group. She is an internationally recognized expert in the validation and benchmarking of biomedical AI and image analysis algorithms, with a particular focus on improving the quality, reliability, and methodological rigor of validation pipelines. Dr. Reinke has played a central role in landmark community initiatives on challenge quality and performance assessment, including the BIAS reporting guideline and the Metrics Reloaded framework. She holds several leadership positions in the international biomedical AI community. She serves as Secretary and incoming President of the MICCAI Special Interest Group on Biomedical Image Analysis Challenges, chairs the MONAI Working Group on Evaluation and Benchmarking, and has served on the challenge teams of major conferences, including MICCAI, MIDL, and NeurIPS. Her contributions to the field have been recognized with several awards, including the Hector Foundation Award.

Synopsis

Rigorous evaluation is essential for understanding the capabilities and limitations of medical AI. With foundation models, evaluation becomes particularly challenging as models become more general and multimodal, are evaluated against various tasks, and capabilities such as reasoning are difficult to assess with conventional metrics. This raises questions not only about how we evaluate models but also how we design datasets and benchmarks that test the capabilities we are interested in. The talk will discuss such challenges and evaluation pitfalls of medical foundation models.