Brain-Computer Interface Enables Paralyzed Patients to Control Digital Avatars for Speech and Movement
Scientists at the University of California, San Francisco, have developed a new brain-computer interface (BCI) that allows paralyzed patients to communicate through both speech and upper-body gestures simultaneously via a virtual avatar. This BCI is the first of its kind to enable both speech and movement activities in a single device. Previous technologies could only facilitate one or the other. The system works by recording brain activity from patients with paralysis using electrodes implanted on the brain's surface as they attempt to speak, make gestures, or do both. Machine learning models then translate this brain activity into attempted phrases or body movements. A personalized, full-body avatar, created using the Unreal Engine video game development platform, performs the decoded speech and gestures in real time. This dual decoding capability allows participants to use speech alone, a gesture alone, or both simultaneously, offering a more natural and expressive form of communication.
