A glitch in the Matrix: Using virtual reality to understand how fish predict the future

Scientists from the Friedrich group have developed a new virtual reality system that allows them to manipulate the sensory environment of adult zebrafish at will, while simultaneously analyzing neural activity. This approach can be used to explore how the brain processes complex sensory inputs and how it uses internal models of the world to control behaviors.

When it comes to studying higher brain functions like learning, social behavior and cognitive behaviors, zebrafish are an appealing, not-too-small, not-too-big, but 'just right' model organism. Their brains are small enough to perform high-resolution calcium imaging microscopy of neuronal activity, and to reconstruct neuronal wiring diagrams of many brain areas. Larger vertebrates, such as mice, have brains so big it is highly challenging to optically access a significant proportion of neurons of interest, while smaller organisms like C. elegans or larval zebrafish have a more limited range of behaviors to study.

However, the range of questions scientists have been able to answer about complex behavior in zebrafish has been limited up to this point by existing experimental methods. The fish need to be somehow head-fixed, so that a microscope can record their brain activity, but they must also remain free enough to 'behave' authentically in response to an environment that we can experimentally control to stimulate the behavior of interest.

Researchers have previously been able to embed larval zebrafish in agarose and use a multiphoton microscope to image neuronal activity evoked by different sensory stimuli. The combination of this approach with electron microscopic reconstructions of neuronal circuits has enabled us, for example, to discover precise wiring patterns among neurons that help distinguish similar smells. In the future, it would be interesting to understand how neuronal networks control complex behaviors like associative learning or social interactions. But such complex behaviors are not yet developed at larval stages, and older zebrafish cannot be restrained in agarose anymore.

A new virtual reality experimental method

In a paper recently published in Nature Methods, the Friedrich group presents a new virtual reality (VR) experimental set up that can resolve this conundrum. In this setup, adult zebrafish are fixed in place by a head bar under a multiphoton calcium imaging microscope. An infrared camera recording the fish's swimming action provides feedback to the virtual environment, so that it can be updated in response to the fish's movements. The fish thus gets the impression that it is moving as expected through this virtual environment. This experimental setup allows neuronal activity to be measured live as the behavior of interest is being performed by the fish.

Fish in the VR showed regular swimming patterns and were attracted to animations of other zebrafish. These basic observations confirmed that fish in the VR behaved normally and reacted to complex stimuli. We then took advantage of our ability to manipulate the VR in ways that we can't manipulate a real fish tank environment. Usually, when the fish turns in one direction, the visual image of the environment on the retina moves in the opposite direction. Here, we programmed the VR system so that when the fish turned, the environment turned the same way for a few seconds.

During reversed sensory-motor coupling, a substantial fraction of neurons in the forebrain showed prominent responses. This activity did not represent classical "sensory responses" or "motor responses" but it represented the mismatch between predicted and actual consequences of motor behavior. Based on experiments performed in mammals it has been suggested that the brain uses an internal model of the world to predict the sensory consequences of its own actions. Such a model must be learned and continuously updated based on experience. The finding that mismatches between the expected and the real sensory input during motor behaviors elicit strong neuronal responses in the forebrain support the notion that internal models of the world are not unique to mammals, but are also prominent in fish. These findings thus provide a starting point to explore how a model of the world is implemented in the neuronal circuitry of the brain and how it is modified by experience. This will ultimately be important when asking whether neurological conditions may involve mismatches between predicted and actual consequences of behaviors that do not normally occur in healthy brains. It also shines light on the fundamental question of how our brain represents the world.

New opportunities in structure-function analysis of neuronal networks

This new VR system could also be used to study other complex behaviors. For example, would a fish attempt to join two virtual fish to its left, or one on its right? One might expect it to join the two on the left, and form a shoal for safety from predators. But what if they were made to appear further away? Such social situations wouldn't be possible to orchestrate in a real fish tank, but this experimental set up makes this, and countless other scenarios, possible.

Rainer Friedrich, speaking about the new questions that can be looked into with this methodology, said: "We are also keen to answer some key questions about the interplay between different types of neurons. Recent work indicates that during learning, both excitatory and inhibitory neurons change their interactions with other neurons. But which type of neurons are in the driver's seat? Do excitatory neurons change first and inhibitory neurons follow, or vice versa? Or do all of neurons change in a concerted fashion? Up until now, we have primarily seen the end point of learning, but the VR now gives us the chance to follow behaviors and neuronal mechanisms in real time during a learning process, thus providing new opportunities to study how the activity of neurons translates into functions and thoughts in the brain."