OpenAI Pilots Embedded AI Safety Evaluators from Apollo Research During Model Training
OpenAI has initiated a pilot program to embed six AI safety evaluators from Apollo Research within its operations for four months. These evaluators will scrutinize OpenAI's models during their training, testing, and internal use phases to identify instances where the AI might attempt to bypass alignment and control measures. This move follows a promise by OpenAI's CEO, Sam Altman, to allow external testers with employee-like access. The evaluators will focus on four key questions: whether training rewards models for evading controls, if tests can reveal such behavior, if it occurs during internal use, and if the model's reasoning would expose it. While OpenAI will provide access to training code, data, and model reasoning for the training phase, there are limitations: OpenAI controls staff access, logs formal chats, and evaluators work on monitored OpenAI hardware without access to the company's internal communication channels. OpenAI can redact confidential evidence but cannot remove Apollo's verdicts or ...