Marr's levels of description

In his 1982 book Vision, David Marr (1945-1980) envisioned a research program for the field of vision research, using a distinction between three complementary levels at which information processing systems may be described:

1. The computational level, at which a system's goal is described
2. The algorithmic level, at which a system's method is described
3. The implementational level, at which a system's means are described

Notice that the teleological connotation of the term goal applies here only insofar as it refers to the objective of someone who designs a system, that is, it does not refer to a teleological aspect of the system itself.

Marr's distinction is a general distinction which can be applied recursively to any part of any system (or to any part of any model thereof). Applied to the visual system or, more generally, to the cognitive system, it yields the following differentiation between topics of interest:

1.	Computational level	GOAL	Mental representations
2.	Algorithmic level	METHOD	Cognitive processes
3.	Implementational level	MEANS	Neural structures

More specifically:

1. At the computational level, the goal of a system is specified in terms of systematicities in the system's output as a function of its input. Applied to the visual system, this level concerns the question of what logic defines the nature of resulting mental representations of incoming stimuli.
2. At the algorithmic level, the method of a system is specified in terms of the mechanisms that transform the system's input into its output. Applied to the visual system, this level concerns the question of how its input and output are represented and how one is transformed in the other.
3. At the implementational level, the means of a system is specified in terms of the hardware of the system. Applied to the visual system, this level concerns the question of how those representations and transformations are neurally realized.

Marr's point was that the three levels of description should all be taken equally seriously, to arrive eventually at a comprehensive theory consisting of three complementary descriptions which, together, explain "how the goal is reached with a method that is allowed by the means". Notice that this pluralist approach to cognition does not reflect so much a metaphysical (or ontological) reading of pluralism -- which assumes that, eventually, a "grand unifying theory" is possible -- but rather an explanatory (or epistemological) reading of pluralism -- which, more pragmatically, focuses on differences and parallels between existing explanations at different levels of description to see if and how they might be combined (see also Metaphors of cognition).


Marr's distinction between the three levels of description stimulates integrative theoretical research. There are no strict borders between the three levels, but the distinction is useful not only to position ideas in the total field of cognitive (neuro)science but also to assess whether ideas formulated at different levels, and thereby perhaps seemingly opposed, might yet be compatible.

This is particularly relevant in view of the often unnecessarily heated discussions in cognitive (neuro)science between:

• Representational approaches, which typically start from (computational) ideas about the nature of mental representations.
• Connectionist approaches, which typically start from (algorithmic) ideas about the transformations from input to output.
• Dynamic-systems approaches, which typically start from (implementational) ideas about the neural realizations.

Roughly, representational theory proposes that cognition relies on regularity extraction to get structured mental representations; connectionism proposes that it relies on activation spreading through a network connecting pieces of information; and dynamic-systems theory proposes that it relies on dynamic changes in the brain's neural state. That is, these approaches indeed focus on different aspects of cognition, but these are in fact complementary aspects, and if one looks beyond the differences in the tools they use to model these aspects, then the conceptual commonalities seem to prevail.

For instance, all three approaches tend to trace their origin back to the early 20th century Gestalt ideas about cognition and about vision in particular. The founding fathers of Gestalt psychology, Max Wertheimer (1880-1943), Wolfgang Köhler (1887-1967), and Kurt Koffka (1886-1941), argued that vision involves a complex interaction between autonomous rules of perceptual organization. They captured this in their motto "the whole is something else than the sum of its parts" (Koffka, 1935, p. 176), and they proposed the Law of Prägnanz as governing principle. This law expresses the idea that the brain, like any physical system, tends to settle in stable states. Applied to vision, Koffka (1935, p. 138) formulated this as follows: "Of several geometrically possible organizations that one will actually occur which possesses the best, the most stable shape". This seminal Gestaltist idea has been implemented in many representational, connectionist, and dynamic-systems approaches --- even though, related to the different levels of description, they use different formal tools to model this idea.

In my research, Marr's distinction between complementary levels of description is a guiding methodological principle (see also Research cycles and Metaphors of cognition).

For an extensive discussion on these issues, see Cognitive Processing 2012


The distinction between the three levels of description may not always be clear-cut, but the following may clarify this distinction further:

• First, the names of the three levels (i.e., computational, algorithmic, and implementational) clearly were triggered by the rise of computers. Computer progammers are well aware of the problem to compute things (level 1) with an algorithm (level 2) that runs on hardware (level 3)

• Second, in cognitive science, Dennett (1978) labeled three description levels similarly by the intentional stance, the design stance, and the physical stance; Glass, Holyoak, and Santa (1979) labeled them similarly by the levels of content, form, and medium; and Pylyshyn (1984) labeled them similarly by the semantic level, the syntactic level, and the physical level.

• Third, the distinction between goal, method, and means was already emphasized by Aristotle (384-322 BC), and indeed, whatever the labels are, the distinction is relevant in many domains, for instance in the kitchen:
  

MEANS		GOAL
METHOD

• Fourth, in biology, the successive findings by Darwin (1844), Mendel (1865), and Watson and Crick (1953) can be said to be findings at different, yet complementary, levels of description:

GOAL	Origin of species (Darwin, 1844) Evolution theory, survival of the fittest
METHOD	Experiments in plant hybridization (Mendel, 1865) Heredity theory, classical genetics
MEANS	Molecular structure of DNA (Watson & Crick, 1953) Molecular biology, modern genetics