Digital Twin of Mouse Brain Predicts Neuron Activity and Connections With Striking Accuracy

Built from brain recordings of mice watching action movies, the AI-powered digital twin simulates real-time neural responses—offering a revolutionary tool to decode how the brain processes vision, forms connections, and learns from experience.

Research: Foundation model of neural activity predicts response to new stimulus types. Image Credit: Isidora Simic / Shutterstock

Much as a pilot might practice maneuvers in a flight simulator, scientists might soon be able to perform experiments on a realistic simulation of the mouse brain. In a new study, Stanford Medicine researchers and collaborators used an artificial intelligence model to build a "digital twin" of the part of the mouse brain that processes visual information.

The digital twin was trained on large datasets of brain activity collected from the visual cortex of real mice as they watched movie clips. It could then predict the response of tens of thousands of neurons to new videos and images.

Digital twins could make studying the brain's inner workings easier and more efficient.

"If you build a model of the brain and it's very accurate, that means you can do a lot more experiments," said Andreas Tolias, PhD, Stanford Medicine professor of ophthalmology and senior author of the study published April 10 in Nature. "The ones that are the most promising you can then test in the real brain."

The lead author of the study is Eric Wang, PhD, a medical student at Baylor College of Medicine.

Schematic of the modeled perspective the animal. a, The retina is modeled as points on a sphere receiving light rays that trace through the origin. An example light ray with polar angle θ and azimuthal angle ϕ is shown in red. b, The light ray is traced to a point m^x, m^y on the monitor. Bilinear interpolation of the four pixels on the monitor surrounding m^x, m^y produces the activation of a point θ, ϕ on the modeled retina. c, 9 examples of the modeled perspective from the left eye of an animal, with 3 horizontal rotations of the optical globe (abduction/adduction) × 3 vertical rotations (elevation/depression). The concentric circles indicate visual angles in degrees.  

Beyond the training distribution

Unlike previous AI models of the visual cortex, which could simulate the brain's response to only the type of stimuli they saw in the training data, the new model can predict the brain's response to a wide range of new visual input. It can even surmise anatomical features of each neuron.

The new model is an example of a foundation model, a relatively new class of AI models capable of learning from large datasets and applying that knowledge to new tasks and new types of data, or what researchers call "generalizing outside the training distribution."

(ChatGPT is a familiar example of a foundation model that can learn from vast amounts of text to then understand and generate new text.)

"In many ways, the seed of intelligence is the ability to generalize robustly," Tolias said. "The ultimate goal - the holy grail - is to generalize to scenarios outside your training distribution."

Mouse movies

To train the new AI model, the researchers first recorded the brain activity of real mice as they watched movies—made-for-people movies. The films ideally approximated what the mice might see in natural settings.

"It's very hard to sample a realistic movie for mice, because nobody makes Hollywood movies for mice," Tolias said. But action movies came close enough.

Mice have low-resolution vision, similar to our peripheral vision, meaning they mainly see movement rather than details or color. "Mice-like movement strongly activates their visual system, so we showed them movies with a lot of action," Tolias said.

Over many short viewing sessions, the researchers recorded more than 900 minutes of brain activity from eight mice watching clips of action-packed movies, such as Mad Max. Cameras monitored their eye movements and behavior.

The researchers used the aggregated data to train a core model, which could then be customized into a digital twin of any individual mouse with some additional training.

Accurate predictions

These digital twins were able to closely simulate their biological counterparts' neural activity in response to various new visual stimuli, including videos and static images. Tolias said the large quantity of aggregated training data was key to the digital twins' success. "They were impressively accurate because they were trained on such large datasets."

Though trained only on neural activity, the new models could generalize to other data types.

The digital twin of one particular mouse was able to predict the anatomical locations and cell types of thousands of neurons in the visual cortex and the connections between these neurons. 

The researchers verified these predictions against high-resolution electron microscope imaging of that mouse's visual cortex, which was part of a larger project to map the structure and function of the mouse visual cortex in unprecedented detail. The results of that project, known as MICrONS, were published simultaneously in the journal Nature.

Opening the black box

Because a digital twin can function long past the lifespan of a mouse, scientists could perform a virtually unlimited number of experiments on essentially the same animal. Experiments that would take years could be completed in hours, and millions of experiments could run simultaneously, speeding up research into how the brain processes information and the principles of intelligence. 

"We're trying to open the black box, so to speak, to understand the brain at the level of individual neurons or populations of neurons and how they work together to encode information," Tolias said.

In fact, the new models are already yielding new insights. In another related study, simultaneously published in Nature, researchers used a digital twin to discover how neurons in the visual cortex choose other neurons to form connections.

Scientists had known that similar neurons tend to form connections, like people forming friendships. The digital twin revealed which similarities mattered the most. Neurons prefer to connect with neurons that respond to the same stimulus - the color blue, for example - over neurons that respond to the same area of visual space.

"It's like someone selecting friends based on what they like and not where they are," Tolias said. "We learned this more precise rule of how the brain is organized."

The researchers plan to extend their modeling to other brain areas and animals, including primates, with more advanced cognitive capabilities.

"Eventually, I believe it will be possible to build digital twins of at least parts of the human brain," Tolias said. "This is just the tip of the iceberg."

Researchers from the University of Göttingen and the Allen Institute for Brain Science contributed to the work.

The study received funding from the Intelligence Advanced Research Projects Activity, a National Science Foundation NeuroNex grant, the National Institute of Mental Health, the National Institute of Neurological Disorders and Stroke (grant U19MH114830), the National Eye Institute (grant R01 EY026927 and Core Grant for Vision Research T32-EY-002520-37), the European Research Council and the Deutsche Forschungsgemeinschaft.