Probing, Modeling and Reprogramming Visual Perception at the Level of Individual Photoreceptors
Abstract
Our perception of the world differs markedly from the physical retinal image. Photoreceptors are punctate, yet perception is spatially continuous. Retinal images are highly dynamic due to fixational drift and saccades, yet perception is stable. Cone cells are discretized and unevenly sampled, yet color perception is continuous and consistent. That this is so makes sense, because the computational goal of the brain is to infer properties of the world from image measurements, not to perceive the retinal image per se. However, the actual neural mechanisms underlying these inferential computations remain largely a mystery. In this project, we seek to elucidate this remarkable neural process by probing, modeling, reprogramming and manipulating visual perception at the level of individual photoreceptors. We aim to do this in the fovea – the most important retinal area, but least studied from a physiological perspective because of its intricacy. This project consists of the following five major lines of investigation. 1. Probing Perception of Form and Color. We will probe how perception disentangles color and form from the population of cone responses, through a new psychophysics methodology of reproducing and micro-adjusting the hundreds or thousands of individual photoreceptor input values underlying a given visual percept. We will study percepts from simple colored rectangles and lines to video of natural scenes. Micro-adjustments of these signals will probe the perceptual contributions of eye motion, cone identity and colorfulness . 2. Characterizing Visual Neural Circuitry. We will probe neural circuits from cones to horizontal cells, retinal ganglion cells, neurons in the lateral geniculate nucleus and neurons in primary visual cortex. We will use optical stimulation of individual photoreceptors in tandem with psychophysics, electrophysiology, two-photon fluorescence and optoretinography (pure optical recording of neuronal activity). These studies will contribute fundamental knowledge about how signaling between neurons is affected by critical aspects of visual neural circuitry- adaptation, kinetics, neural noise, lateral inhibition and which signal-aggregating neural pathway is involved. 3. Theory, Modeling and Simulation of Visual Perception. We will develop computational models of visual perception from photoreceptors to cortical circuits. Our recent work demonstrated (1) that retinal image motion improves acuity, and (2) a computational model for how neurons in cortex could build up a high acuity representation of an object by moving the cone mosaic over the object. Here, we will incorporate known neural circuitry structure and function, and extend this model to study how the brain disentangles form and color. We aim to corroborate perceptual findings and generate hypotheses for further experiments. 4. Reprogramming Visual Perception. Informed by theory and modeling, we will probe neural plasticity to generating novel percepts from new sensory input. For example, we aim to program retinal stimulation patterns that elicit trichromacy in a dichromat and tetrachromacy in a trichromat. If successful, these experiments will demonstrate a potential for fundamental richer information transfer to the brain via the retina. One potential application is to add thermal spatial awareness atop normal color experience. Another possibility is injecting symbolic data into the brain without interrupting regular vision. 5. Science and Engineering of Retinal Imaging and Stimulation. We will support the proposed experiments with low-risk but extensive engineering upgrades to our existing technologies for retinal imaging and stimulation. In addition, we will pursue high-risk but high reward basic research in imaging science, directed towards ultra-fast, programmable stimulation of thousands of individual cone cells during natural viewing.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Mar 07, 2023
- Source ID
- FA95502110230
Entities
People
- Ren Ng
Organizations
- Air Force Office of Scientific Research
- United States Air Force
- University of California Regents