Johns Hopkins and FDA Develop Tool to Identify Bias in Medical AI Training Data
Researchers at Johns Hopkins University, in collaboration with the U.S. Food and Drug Administration (FDA), have developed a tool named Generalized Attribute Utility and Detectability-Induced Bias Testing (G-AUDIT). This tool is designed to identify hidden biases in datasets used for training medical artificial intelligence (AI) systems. The tool examines training data to detect subtle patterns that could lead AI models to make incorrect conclusions, a phenomenon known as the 'Clever Hans' problem. This issue arises when AI models learn to associate irrelevant signals with clinical outcomes, such as associating the presence of mascara with gender rather than biological features. The G-AUDIT tool aims to address this by flagging metadata that could mislead the model, thus intervening earlier in the AI development pipeline. The research has been published in the journal npj Digital Medicine. 
