Smart processing

Transparallel processing means that many items are processed simultaneously as if only one item were concerned (for a natural example, see Pencil selection). In computer science, this would be called a case of smart processing. That is, in computer science, one often is faced with problems that, at first glance, seem to require an amount of computing time that exceeds the time span of this universe. Smart processing then refers to algorithmic methods that enable a computer to solve such problems in a tractable amount of time. Our brain is not a computer, but it yet seems to do pretty smart processing. Ideas about smart processing are therefore relevant not only to computer science but also to cognitive science.

If, at first glance, a job seems to require an exponential number of computing steps, say in the order of 2^N steps for an input of size N, then a smart process would typically be a process that requires only a polynomial number of steps, say in the order of N² steps. In the case of transparallel processing, the processing of an exponential number of items is even reduced to the processing of only one item. In general, smart processing depends heavily on the way in which to-be-processed information is represented in a data structure, and also this aspect seems relevant to cognitive science.

Hence, smart processing has to do with specific forms of processing as allowed by specific data structures.


Forms of processing

In principle, the following forms of processing can be distinghuised:

• subserial processing --- items are processed one after the other by many processors;
• serial processing --- items are processed one after the other by one processor;
• parallel processing --- items are processed simultaneously by many processors;
• transparallel processing --- items are processed simultaneously by one processor.
  

Many real-life situations involve some hybrid combination of different forms of processing. For instance, at the checkout in a supermarket, the cashiers work in parallel, each cashier processes customers seriallly, and the customers process their carts subserially (i.e., the carts are presented one after the other, each cart by its own owner).

Just as parallel processing, also transparallel processing implies that items are processed simultaneously. However, whereas parallel processing needs many processors, transparallel processing needs only one processor. This distinction may be clarified further in computer terms as follows:

• Parallel processing is primarily a hardware solution to speed up things: Instead of assigning a job to one processor, the job is divided into subjobs that can be done simultaneously by different processors (various variations on this theme have been invented). This reduces the amount of time needed to complete a job but not the amount of work in, say, man-years.

• Transparallel processing is primarily a software solution to speed up things: Instead of assigning a job to one processor or delegating subjobs to different processors, the job is reduced beforehand by collapsing subjobs into one subjob (e.g., in the Pencil selection example, by considering one bundle of pencils instead of many separate pencils). This not only reduces the amount of time needed to complete a job but also the amount of work in man-years.

Above, one may miss the notion of distributed processing but this notion actually belongs to the realm of data structures.


Data structures

In computer science, distributed processing usually means that a job is divided into subjobs that are delegated to a number of processors. These processors then may or may not work in parallel but more important is that (a) each processor processes only a part of the total information needed to complete the job, and (b) that the processors are linked to form a network that, as a whole, deals with the entire job. In other words, distributed processing actually stands for processing in a distributed representation of information. Our brain, being a network composed of many interconnected neurons (the "processors"), most probably does a lot of distributed processing.

Hence, in both computer science and cognitive science, the crucial point of distributed processing is the usage of a network that regulates an interaction between bits of information. Such a network can be implemented in various ways. First, the bits of information can be taken to be represented by either the nodes (localist approach, as in the World Wide Web) or the links (distributed approach, as in a road map). Second, the network can be taken to consist of either many interacting processors (one in each node) or just bits of information which one processor operates on (to regulate the interaction).

For instance, in the 1980s, parallel distributed processing (PDP) models became popular in cognitive science (first localist approaches, then distributed approaches). In theory, a PDP model models a cognitive process by an activation flow that spreads through the links in a hardwired network with nodes as parallel processors of incoming and outgoing activation. In practice, a PDP network is hardly ever really built but is usually simulated on a computer that operates on a software version of the network. The computer then of course has to do serially what is done in parallel in the hardwired network, that is, the computer actually performs serial distributed processing, but this does not alter the crucial processing principle of interacting bits of information.

In computer science, already since the 1950s, serial distributed processing has been quite common in smart algorithms. A classical example is Dijkstra's (1959) shortest path method (SPM). To select a shortest path from among an exponential number of paths, the SPM does not check every possible path separately, but it uses a distributed representation of the paths (as in a road map) in a computer algorithm that selects a shortest path in a polynomial number of steps. The SPM is also an example of a serial distributed processing method that can be translated into an otherwise equivalent method that performs parallel distributed processing in a hardwired network (see Slimy, Hilly, and Pixy).

Hence, the distinction between serial and parallel processing may be relevant for practical time purposes, but it neither affects information-processing principles nor the amount of work to be done. What does affect these things is the usage of distributed representations: it thrives on the processing principle of interacting bits of information and it typically implies a reduction from order 2^N to order N² in the number of subjobs to be done.


Transparallel processing by hyperstrings

The foregoing ideas about forms of processing and data structures converge in the idea of hyperstrings. A hyperstring is a distributed representation of up to 2^N strings that can be searched for regularities as if only one string of length N were concerned. Thereby, hyperstrings allow for transparallel processing as defined above. This affects information-processing principles not only because hyperstrings are special distributed representations (see Hyperstrings) but also because transparallel processing is a form of processing that transcends the traditional distinction between serial and parallel processing (see Pencil selection).

Summarizing, in the number of subjobs to be done,

Furthermore, conventional PDP models usually start from a fixed network in which, beforehand, all possible outputs for all possible inputs are represented. In SIT, that is, in the context of human pattern encoding based on regularity extraction, however, hyperstrings are networks that are created on the fly to represent only a subset of all possible outputs for only the input at hand. This suggests, in cognitive science, that the more or less fixed neural network in the brain allows for flexible cognitive networks that change with changing input. This seems to agree with the finding, in neuroscience, of transient neural assemblies which signal their presence by synchronous firing of the neurons involved; perhaps, this synchronization is a manifestation of transparallel processing.

For a formal account of hyperstrings, see Proceedings of the National Academy of Sciences USA 2004.