Going beyond statistical significance: Understanding effect sizes in terms of their predictive utility
When and Where
Speakers
Description
A lot of fMRI work is ultimately motivated by the hope of improving real-world outcomes in mental health and related areas. Yet, despite thousands of statistically significant findings, we still have very few biomarkers or prediction models that are useful at the individual level. One reason is that our analyses are optimized for statistical significance (p-values), not for practical significance (effect sizes). We simply lack principled ways to interpret effect sizes. In this talk, I will introduce predictive utility analysis as a way to interpret effect sizes in terms of their real-world predictive value. To illustrate this approach, I will use E2P Simulator (www.e2p-simulator.com), an open-source interactive web tool that we recently developed for this purpose. E2P Simulator converts familiar effect-size metrics (e.g., Cohen’s d, Pearson’s r) into discriminative ability (ROC-AUC, sensitivity, specificity), predictive value (PPV, NPV, PR-AUC), and clinical utility metrics (Net Benefit), while explicitly accounting for measurement reliability and base rates (e.g., prevalence, event rates). Using concrete examples from psychiatry and neuroimaging, I will illustrate how this framework helps judge when an observed effect has a realistic path to real-world translation.
About ToNI Users Meetings
The bi-weekly ToNI Users Meeting forms the core of our community of practice — a friendly, open forum for everyone conducting MRI research at ToNI. In an informal, seminar-style format led by trainees from across departments, users present new project ideas, discuss methodological questions, share results, or highlight new trends in neuroimaging. Faculty experts and the MR Physicist join regularly to provide scientific and technical input, fostering constructive exchange and collaborative problem-solving. The goal: to help users help each other, building shared knowledge and ensuring the best possible data and outcomes from every ToNI study.