paraafPeter A. van der Helm Demo link
About Teaching Research Publications Presentations


Structural information theory and its applications



Theory

In the 1960s, Leeuwenberg initiated structural information theory (SIT). SIT began as a coding model of visual pattern classification (demo) that, in interaction with empirical research, developed into a competitive theory of perceptual organization. Nowadays, SIT includes comprehensive models of amodal completion (demo) and symmetry perception (demo). Furthermore, in object perception, SIT proposes an integration of viewpoint-independent and viewpoint-dependent factors quantified in terms of object complexities (demo).

Central to SIT is the simplicity principle, which implies that the visual system is assumed to prefer the simplest interpretation among all possible interpretations of a stimulus. To make predictions, the interpretations are represented by symbol strings, and the symbol string with the overall simplest code is taken to specify the preferred interpretation. A simplest code is a symbol representation that enables the reconstruction of the stimulus using a minimum number of descriptive parameters; it is obtained by capturing a maximum amount of regularity in the symbol strings, and it implies a hierarchical stimulus organization in terms of wholes and parts.
Emanuel Leeuwenberg
Emanuel Leeuwenberg

See Acta Psychologica 2003 for a special issue in his honor, on the occasion of his 65th birthday.

Historically, SIT's simplicity principle is a descendant of Hochberg and McAlister's (1953) minimum principle which, in turn, is a descendant of Koffka's (1935) Gestalt law of Prägnanz. Koffka referred to the natural tendency of physical systems to settle into minimum energy states to which, in the case of the human visual system, preferred interpretations might correspond. Hochberg and McAlister casted this Gestalt idea in terms of representational compactness by replacing minimum energy with minimum descriptive information. SIT added a concrete coding language specifying the so-called transparent holographic regularities to be captured in descriptive codes. For the mathematical foundation of SIT's coding language, see Journal of Mathematical Psychology 1991, and for the psychological foundation of the transparent holographic nature of visual regularity, see Psychological Review 1996 and Psychological Review 2004.

SIT's simplicity principle concurs with the minimum description length principle in the mathematical domain of algorithmic information theory (AIT) that, also in the 1960s, was initiated by Kolmogorov and Solomonoff. Until the 1990s, SIT and AIT evolved independently. There are differences between SIT and AIT:
For the rest, however, SIT and AIT share many starting points and objectives. SIT and AIT can be said to present a viable alternative to Shannon's (1948) probabilistic information theory and to the classical Helmholtzian likelihood principle which, in vision, assumes that the preferred interpretation of a proximal stimulus is the one with the highest probability of specifying the actual distal stimulus.

Just as Shannon's theory, the Helmholtzian likelihood principle presupposes knowledge about probabilities in terms of, for instance, frequencies of occurrence of objects in the world. Such probabilities, however, are often hardly quantifiable, if at all (demo). In SIT and AIT, this problem is circumvented by turning to precisals, that is, artificial probabilities derived from the length of shortest descriptive codes. AIT has shown that precisals might well be reliable alternatives for the often unknown real probabilities. In vision, this paved the way for a more detailed comparison of SIT's simplicity principle and Bayesian implementations of the likelihood principle (demo).

This comparison revealed that precisals and real probabilities may be far apart for the viewpoint-independent factors (Bayesian priors) but seem close for the viewpoint-dependent factors (Bayesian conditionals) that are decisive in the everyday perception by a moving observer. This implies that both the simplicity principle and the likelihood principle may have guided the evolution of the human visual system, the difference being that the likelihood principle assumes the human visual system is a special-purpose system, whereas the simplicity principle assumes it is a general-purpose system. That is, the likelihood principle starts from optimally veridical vision in only the one world in which we live, whereas the simplicity principle suggests that fairly veridical vision in many worlds is a emergent side effect of the preference for simplest interpretations. For an extensive discussion of these issues, see Psychological Bulletin 2000.

In Marr's (1982) terms, SIT began as a theory at the computational level of description (demo). Just as Bayesian implementations of the likelihood principle, for instance, SIT models vision as if it considers all possible interpretations before it selects a preferred interpretation. Thereby, SIT models process outcomes rather than process mechanisms. Nowadays, however, SIT also includes process models. For instance, see Psychological Review 1999 for the so-called holographic bootstrapping mechanism in the detection of visual regularity (demo), and see Proceedings of the National Academy of Sciences USA 2004 for the so-called transparallel processing mechanism in the selection of simplest interpretations.

This transparallel processing mechanism relies on so-called hyperstrings (demo), which are distributed representations that enable the processing of an exponential number of codes as if only one code were concerned (demo). This goes beyond the parallel distributed processing (PDP) mechanism in so-called "neural" network models (demo), and has an additional storage advantage: PDP models presuppose that all possible outcomes for all possible inputs are stored beforehand in a network, whereas SIT's transparallel processing model performs an on-the-fly construction of only a subset of all possible outcomes for only the input at hand (demo).

The latter suggests, in cognitive science, that the more or less fixed neural network in the brain allows for flexible cognitive networks that change with changing input. This seems to agree with the finding, in neuroscience, of transient neural assemblies which signal their presence by synchronous firing of the neurons involved. Perhaps, this synchronization is a manifestation of transparallel processing.

For a pdf-presentation on this theoretical framework, see Visual regularity or its printable handout.



Applications

The conglomerate of ideas within SIT has found societal applications in art science and visual ergonomics. This led, for instance, to traffic reconstructions yielding safer roads, bridges, and tunnels (demo). Furthermore, during the past decades, it has been applied to a wide range of topics in vision science. These topics include: