Two monkeys with existing brain implants received stimulation patterns chosen by a computer model, and the patterns shifted how the animals perceived objects in predictable ways. The model was built for a visual prosthesis by the NeuroAI Lab of Martin Schrimpf at EPFL, and the live trials were run by researchers in Amsterdam. The preliminary results were presented in April at the International Conference on Learning Representations.
The limit of current cortical implants
Visual prosthetics come in several forms. Retinal prosthetics sit on the retina. Optic nerve prosthetics are used when the retina is too damaged for an implant, and they stimulate the nerve instead. When neither the retina nor the optic nerve can take an implant, cortical prosthetics are the option. These bypass both structures and use electrodes to stimulate the visual cortex, effectively drawing images onto it.
The approach has so far been restricted to lower-level regions of the brain. In those regions it is only possible to project flashes of light and simple shapes. Hardware adds a second constraint: stimulating several areas at once takes multiple electrodes, and only a certain number can be placed in one area.
Johannes Mehrer, a scientist in the NeuroAI Lab who led the research, said the images these systems produce are limited in complexity. Existing approaches, he said, cannot elicit the percept of a more complex object such as a house or a car. The motivation, he said, is the number of people whose visual deficits are irreparable because damage somewhere along the visual processing stream, starting with the retina, cannot be fixed.
Targeting higher-level visual regions
Higher-level visual regions of the brain handle the processing of complex objects. They could therefore be a target for a new generation of prosthetics that evoke faces, houses and other objects. The difficulty is access. Researchers do not know exactly where or how to stimulate these regions.
The EPFL team used an artificial neural network of a type called a topographic neural network. The network tests patterns of stimulation in the higher-level regions and simulates their outcomes. Mehrer said this allows many simulations with different combinations of parameters, work that would otherwise consume a great deal of experimental time and money.
The EPFL researchers worked entirely on computers. They set up the model to select the best combination of images and the specific stimulation pattern to go with each. A team in Amsterdam then tested the model's predictions on two of their monkeys, which already had implants for other experiments unrelated to EPFL.
What the monkey trials showed
Schrimpf said the model proved quite efficient at predicting which stimulation pattern would have a strong effect on the monkeys' behavior in visual object recognition. He said the model can select images, but that the more important function is the reverse: given an image, it can identify the optimal stimulation pattern for a desired behavior.
The result has a defined scope. The researchers showed that they can shape object perception. When a visual stimulus is presented, they can bias how the brain represents it. They cannot yet create object perception from nothing.
Schrimpf described the experiment this way: the monkey had already seen an image, and the team was able to distort it to change perception in somewhat predictable ways. The next step is to stimulate the cortex when no visual stimulus is present. The larger goal, he said, is to evoke a percept from scratch, so that someone sees something meaningful even when their eyes are not delivering a usable image.
Hearing is next
The team plans to test whether the same modeling approach works for auditory stimulation. The work is funded through a grant from the Horton Health Foundation. Schrimpf said cochlear implants are valuable but imperfect and do not fully restore auditory processing. The lab intends to build topographic models that predict what stimulation does to neural activity in auditory processing.
Analysis: what this changes and what it does not
The main contribution is a method rather than a device. Cortical prosthetics have been held back by the small set of percepts that low-level stimulation can produce. This work moves the target to higher-level regions and supplies a way to search the very large space of stimulation patterns in software before any animal is involved. Because the simulations cost far less than experiments, that search can be repeated across many parameter combinations.
The comparison with existing approaches is straightforward. Retinal and optic nerve implants depend on part of the visual pathway being intact. Current cortical implants avoid that dependency but deliver light flashes and simple shapes. The model-guided approach keeps the cortical route and aims at richer content. It does not yet deliver that content, since the demonstrated effect is a bias on an image the animal was already viewing.
Several points remain unproven. The evidence comes from two monkeys that were sighted and already implanted, and the results are described as preliminary. No result has shown that a percept can be produced with no visual input, which is the condition that matters for a blind user. The source also does not report human testing. Whether predictions that hold in monkeys carry over to human cortex is an open question that these trials cannot answer.
The hardware limit on electrode counts also persists. The model can choose better stimulation patterns, but it does not remove the cap on how many electrodes fit in one area. How much a well-chosen pattern can compensate for that cap is not yet established.
For robotics and neurotechnology readers, the notable point is the role of the model as a design tool for the interface. It selects both images and stimulation patterns, and it works backward from a desired behavior to the pattern that should produce it. If the planned from-scratch stimulation succeeds, the same approach could be tested for hearing, as the lab intends. Until then, the demonstrated gain is a tool for predicting and shaping perception, not a restored sense of sight.