New Robot Steering Framework 'ReStruct' Enables Natural Language Control Without Retraining
Researchers have developed 'ReStruct,' a novel zero-shot method that allows robots to be steered by natural language preferences at inference time without requiring retraining. This framework addresses a significant challenge in robotics: adapting learned policies to user preferences or constraints not anticipated during initial training. Existing methods often fail either by requiring extensive fine-tuning (end-to-end policies) or by producing logically valid but physically infeasible plans (neuro-symbolic methods). ReStruct decomposes a visuomotor policy into a high-level state-machine skeleton and a low-level residual controller. A vision-language model (VLM) interprets user preferences, grounds them within the robot's learned symbolic structure, and refines this structure if necessary. The system then compiles these preferences into a temporal-logic automaton, which reconfigures the task structure. Crucially, ReStruct updates the skeleton's action priors to ensure physically-aware control, even with th...