Anthropic Implements Stricter Usage Policy for AI-Driven Physical Systems
Anthropic's updated Usage Policy, effective November 12, mandates new safety requirements for equipment controlled by its Claude AI that can cause injury without direct human approval. The policy stipulates that a qualified individual must be able to observe and stop the equipment at any time. Furthermore, the equipment must automatically stop or enter a safe state if the human operator intervenes or if the connection to Anthropic's services is lost. Crucially, safety limits, such as speed or force, must be enforced by the equipment itself or an independent controller, not solely by the AI model's output. This change addresses vulnerabilities identified by security researchers, who have demonstrated that LLM-controlled robots can be 'jailbroken' through prompt injection attacks. The policy applies to six categories of equipment, including those that move through shared spaces, apply injurious force, or control hazardous materials. Exclusions include consumer devices within built-in safety limits, plans rev...
