AI Alignment Techniques Require Strict Separation of Training and Evaluation Data
A recent argument emphasizes the critical need for AI alignment techniques to maintain a strict separation between the data used for training models and the data used for evaluating them. This approach mirrors traditional machine learning practices, where 'train-test splits' are considered fundamental. The author contends that if AI models are trained using all available alignment monitors, these monitors lose their effectiveness in detecting covert misalignment. This practice leaves researchers without a reliable test set to verify the model's behavior before its deployment. The recommendation is for frontier AI companies to establish and maintain transparent, held-out alignment test sets and to clearly differentiate between their training and testing methodologies.