Statistical Models of the Early Visual System

Yan Karklin, Carnegie Mellon University

The visual system is able to perform a variety of complex tasks given only a noisy, wildly varying input at the retina. It can perceive shapes, recognize textures and objects, and infer scene properties in a remarkably fast and reliable way and under all sorts of adverse conditions. How does the brain solve these problems? What sort of representations does it form and what computational principles underlie this processing? Using electrophysiology to map out neural responses to visual stimuli has yielded a wealth of data, but without computational theories, it is difficult to interpret the recorded neural activities, especially for increasingly specialized neurons deeper in the visual processing hierarchy.


One theoretical approach is to assume that the brain has evolved to process incoming information as efficiently as possible. This computational principle, known as “efficient coding”, states that optimal representations should reduce redundancy and encode only the independent elements of the input. Because the optimal coding strategy is closely related to the statistical properties of the input, good models of natural sensory environments (the brain’s input) should provide insight into the computational mechanisms of the visual system.


Early work used statistical models adapted to natural sensory input to make predictions about neural properties in the early visual system. Using only the notion that the brain’s task is to extract useful information from a noisy scene and represent it efficiently, it has been possible to deduce theoretically optimal representations for low-level visual features. These predictions are remarkably similar to physiologically measured properties of simple cells in the primary visual cortex. I will describe this work and its limitations, and then discuss our research into hierarchical extensions of the linear statistical models that capture higher-order dependencies and non-stationary statistics of natural images.

Abstract Author(s): Yan Karklin, Michael S. Lewicki