This AI-powered brain implant hints at the future of human interaction

FastCompany: For the first time, a single brain implant can now read a paralyzed person’s attempts to speak and gesture at the same time, turning them into text and the gestures of a digital avatar. According to the research paper published in Nature, the device uses artificial intelligence models to read the brain and transform its impulses into speech and body language. Current brain-computer interfaces (BCI) can decode only one thing at a time. You can either reliably translate speech or you can reliably translate gestures, but not both.

They stumble because the brain areas involved in talking about something while moving your hands as you speak partly overlap. Scientists from the University of California, San Francisco (UCSF), have now showed that their new system handles speech and gestures together thanks to artificial intelligence. Right now, it can handle only small vocabularies and two participants for simultaneous decoding. It’s also imperfectly accurate. But it’s a big step forward, because all humans communicate not only with language but also with how our bodies move.

Hand gestures, like facial expressions, are an integral part of the experience of interacting with other people. But this is only the first step toward a brighter future. “Our longer-term goal is to move toward what we think of as a ‘whole-body BCI’: a single brain interface that can restore multiple forms of communication and movement,” Samantha Brosler, one of the lead authors of the research paper, tells Fast Company .

“In principle, the decoded speech and movement signals could be used to control a virtual avatar, other assistive technologies, or a physical robotic surrogate.” [Image: Todd Dubnicoff (video editing)/UCSF Weill Institute/Chang Lab/UC Berkeley] Brosler, a researcher in the University of California, Berkeley–UCSF graduate program in bioengineering, tells me that, ultimately, they want to give people with paralysis more flexibility in how they communicate and interact with the world, “rather than limiting them to controlling one function at a time.” Since the 2000s, scientists have been developing BCIs capable of triggering certain robotic arm

movements, but Brosler says they want to move past those predefined gestures and develop systems that continuously control movements across the whole body in a natural way. It will mean making the decoding of brain signals faster and more accurate, and studying whether real-time sensory feedback can make the controlling of avatars or robots feel more natural and intuitive. In her view, the study is an early base on which the flexible, multifunctional control that might make this possible can eventually be built. Two signals, one cortex When paralysis interrupts the path between brain and body, the brain still issues commands.

This is what allows scientists to build BCI implants capable of capturing the electrical signals that represent those commands. The implant is a high-density electrocorticography array designed to catch those commands. Imagine a small flexible sheet studded with a regular grid of tiny metal dots, laid directly on the surface of the brain over the areas that control movement and speech. Hand and arm attempts show up in one region of the grid, head and eye movements in another, and speech and mouth movements in the lower part. But speech and gestures areas overlap in the brain, which makes things confusing for computers.

Article truncated for readability. Read the full piece →

The development of an AI-powered brain implant represents a groundbreaking advancement in communication technology, which has significant implications for user interaction and brand strategy.