AI Coach Improves Interview Practice for Formerly Incarcerated Individuals at U.S. Reentry Nonprofit
The Center for Employment Opportunities (CEO), the largest reentry employment organization in the United States, has significantly improved its AI-powered mock interview coach, ACE. This AI coach assists formerly incarcerated individuals in preparing for job interviews across more than 30 U.S. cities. Initially, while participants liked ACE, there was no systematic way to evaluate its effectiveness. Researchers from The Bike Shop, an AI research lab, partnered with CEO to develop an evaluation system using a second AI, an 'LLM-as-judge,' to score coaching calls against a rubric of good coaching. This system identified key weaknesses in the original ACE, such as a lack of dialogue, low participant practice rates, and over-praising weak answers. Following these findings, a new version of the AI coach was designed and tested. An A/B test revealed substantial improvements: 96% of participants now attempt an improved answer after feedback, compared to 46% previously; the AI coach successfully limited feedback l... 
