Toward a Unified Science of the Mind-Brain
Patricia Smith Churchland
1986
A Bradford Book The MIT Press Cambridge, Massachusetts London, England
Pages in facsimile (not searchable)
(Pages in searchable html)
Chapter 10
What we require now are approaches that can unite basic neurobiology and behavioral sciences into a single operational framework.
(Dominick Purpura, 1975)
10.1 Introduction
An enormous amount is known about the structure of nervous systems. What is not understood is how nervous systems function so that the animal sees or intercepts its prey, remembers where it cached nuts, and so forth. We are beginning to understand the behavior of an individual neuron - its membrane properties, the spiking properties of its axon, the synaptic phenomenology, its patterns of connectivity, the transport of intracellular materials, its metabolism, and even something of its embryological migration and development. To be sure, many unanswered questions remain, and major breakthroughs are yet to be made, but the neuron is not a smooth-walled mystery. Neuroscientists have considerable confidence at least about what sort of research would lead to answers to outstanding questions, and they have a general picture of what, more or less, the answers will look like. On the other hand, the state of theory of how ensembles of neurons result in an owl's being able to intercept a zigzagging mouse, for example, or how any creature visually recognizes a given object, is markedly different. Here there is no widely accepted theoretical framework, nor even a well-defined conception of what a theory to explain such things as sensorimotor control or perception or memory should look like.
Theorizing about brain functions is often considered slightly disreputable and anyhow a waste of time-perhaps even "philosophical." A neuroscientist randomly plucked out of the crowd at the Society for Neuroscience meetings and asked about the role of theories in the discipline will likely answer with one or all of the following: (1) "The time for theories has not yet arrived, since not enough is known about the structural detail," (2) "What is available by way of theory is too abstract, is untestable, and is anyhow irrelevant to experimental neuroscience," (3) "You cannot get a grant for that sort of monkey business." (For a sophisticated version of (1), see Selverston 1980.)
One cannot escape having some sympathy with these sentiments. If one is going to do research, one must attract grants and get results. And at least if one is doing experiments, the techniques, the methods, and the procedures are often clear enough. One can lesion, implant electrodes, perfuse neurotransmitters, and so forth. If theorizing is the task, however, the techniques and methods are discouragingly amorphous. There is no reliable routine or well-honed method-just the rather empty exhortation to "have good ideas." There is, moreover, a substantial risk in spending time and effort in understanding a theory well enough to figure out ways to test it, and then spending yet more time and effort in conducting the experiments. If the theory turns out to be a flop, and in the absence of a mature paradigm it well may, then the investment may be a career disaster. So the decision to adopt a policy that says "Leave theorizing to the theorizers" is by no means irrational.
On the other hand, the value of theory is that it motivates and organizes experimental research, and good theory opens doors to important experimental results. By shunning theory, one runs the risk that the data-gathering may be random and the data gathered, trivial. This is not an idle worry in the current state of neuroscience, for as a number of scientists have pointed out, many grant proposals are not motivated by a genuine hypothesis at all. It sometimes happens that a piece of research is undertaken not in virtue of a larger program for which the results are important, but because the researcher has mastered a certain technique, and there are always more measurements he can make. The technique comes to structure and govern the research program rather than the other way around. The justification offered for such research is the "maybe-mightbe" twostep: "If . . . then maybe . . . , and then my results might be important."
The consequence of research thus motivated is a huge stockpile of data whose relevance is God knows what. The idea that all results are important or will some time be found to be important is an example of the inductivist fallacy. According to the inductivist strategy, one first gathers all the data, and only then can one set about theorizing. Progress in science is seldom made that way, but is made instead by approaching Nature with specific questions in mind, where the questions are spawned in the context of a hypothesis (Popper 1935, 1963). Consider, for example, how Crick and Watson figured out the molecular structure of DNA. Not, evidently, by first gathering all data and then letting them fall into place, but rather by trying a hypothesis, finding it in ruins, then dreaming up another hypothesis and testing it, and so on and inventively on, until a fit was found. Many of the data gathered in a random data-gathering venture will be useless. It is therefore troubling that a fast sampling of researchers at the poster sessions of the Society for Neuroscience meetings will yield at least several unblushing instances of the "maybe-mightbe" twostep.
In a general sense the best experiments are those whose results shake loose important information, but to design such an experiment one must know what are the right questions to ask. The more coherent and rich the available theoretical framework, the greater the potential for putting to Nature the right questions. Once a theoretical framework matures, the symbiosis between the theory and the experiments causes both to flourish, and the better the theory, the better the questions put to experimental test. Physics and genetics are renowned illustrations of the fruitful symbiosis of theory and experiment.
Moreover, it is an illusion to suppose that experimental research can be completely innocent of theoretical assumptions. So long as there is a reason for doing one experiment rather than another, there must be some governing hypothesis or other in virtue of which the experimental question is thought to be a good question, and some conception of why the experiment is worth the very considerable trouble. There must, that is, be some sense of how the results are significant for the larger picture of how the brain works (Kuhn 1962). This conglomeration of background assumptions, intuitions, and assorted preconceptions, however loose and vague, is the theoretical backdrop against which an experimenter's research makes sense to him. What is wanted, therefore, is not no theory but rather good theory-testable, coherent, richly ramified theory. The dearth of fleshed-out, testable theory is therefore something to be rectified, not patiently endured. (This point has by no means gone unnoticed in the neurosciences. See, for example, the commentaries on Selverston 1980, especially Calabrese 1980, Hoyle 1980, and Lent 1980.)
A third and rather obvious point is also relevant. Theories will not of their own accord waft up out of the data. If we are to explain how ensembles of neurons succeed in, say, coordinating movement, then we need a functional story that will explain how the structure works. The structural details are, to be sure, crucially important, but even when they are known, there remains the problem of accounting for how the ensemble works. And the function of the ensemble cannot be just read off the data concerning the participating neurons since, among other things, the interactions between components are nonlinear. Whatever it is that ensembles do, it will not look like what components do, nor will it be a summation of what components do. (See also Bullock 1980, Davis 1980.)
How to characterize the mathematical relationships between the response profiles of the input and output ensembles is not something that in effect falls out of an array of data, though it may well be inspired by it. Theories are interpretations of the data; they are not merely generalizations over data points: Additionally, and this cuts against the idea that a complete collection of the data must be in place before theorizing is useful, whether some aspect of neuronal or ensemble business is a "relevant structural detail" may in fact be recognized only under the auspices of a theory that purports to explain ensemble function. For example, unless you think that DNA is hereditary material, you will not think the organization of nucleotides is relevant to determining the phenotype.
Although there is an undercurrent of reticence regarding theory in neuroscience, nonetheless there is a growing recognition of the need for theorizing. If neuroscience is to have a shot at explaining-really explaining-how the brain works, then it cannot be theory-shy. It must construct theories. It must have more than anatomy and pharmacology, more than physiology of individual neurons. It must have more than patterns of connectivity between neurons. What we need are small-scale models of subsystems and, above all, grand-scale theories of whole brain function.
The cardinal background principle for the theorist is that there are no homunculi. There is no little person in the brain who "sees" an inner television screen, "hears" an inner voice, "reads" the topographic maps, weighs reasons, decides actions, and so forth. There are just neurons and their connections. When a person sees, it is because neurons, individually blind and individually stupid neurons, are collectively orchestrated in the appropriate manner. So much seems obvious, and even a brief immersion in the neurosciences should proof one against the seductiveness of homuncular hypotheses.
Surprisingly, however, homunculi, or at least the odor of homunculi, drift into one's thinking about brain function with embarrassing frequency.
Part of the explanation for the enduring presence of homuncular preconceptions is that folk psychology still provides the basic theoretical framework within which we think about complex behavior. Unless warned off, it insinuates itself into our thinking about brain function as well. In a relaxed mood, we still understand perceiving, thinking, control, and so forth, on the model of a self-a clever self that does the perceiving and thinking and controlling. It takes effort to remember that the cleverness of a brain is explained not by the cleverness of a self but by the functioning of the neuronal machine that is the brain. (See also Crick 1979.)
Crudely, what we have to do is explain the cleverness not in terms of an equally clever homunculus, and so on in infinite regress, but in terms of suitably orchestrated throngs of stupid things (Dennett 1978a, 1978b). In one's own case, of course, it seems quite shocking that one's cleverness should be the outcome of well-orchestrated stupidity. The sobering reminder here is that so far as neuronal organization is concerned, there appears to be no rationale for giving a system conscious access to all-or even very many-of the brain's states and processes.
10.2 In Search of Theory
What is available by way of theory? Are there theories that have real explanatory power, are testable, and begin to make sense of how the molar effects result from the known neuronal structure? Less demandingly, are there theoretical approaches that look as though they will lead to fully fledged theories?
The fast answer is that a lot of very creative and intelligent work is going on in a number of places, but it is uneven, and it is difficult to determine how seminal most of it is. I began scouting the theoretical landscape with neither a clear conception of what I was looking for nor much confidence that I should recognize it if I found it. Most generally, I was trying to see if anywhere there was a kind of "Galilean combination": the right sort of simplification, unification, and above all, mathematization - not necessarily a fully developed theory, but something whose explanatory beginnings promised the possibility of real theoretical growth.
In coming to grips with the problems of getting a theory of brain function, I had to learn a number of general lessons. First, there are things that are advertised as theories but are really metaphors in search of a genuine theoretical articulation. One well-known example is the suggestion first floated by Van Heerden (1963) that the brain's information storage is holographic. (See also Pribram 1969.) Now the brain is like a hologram inasmuch as information appears to be distributed over collections of neurons. However, beyond that, the holographic idea did not really manage to explain storage and retrieval phenomena. Although significant effort went into developing the analogy (see, for example, Longuet-Higgins 1968), it did not flower into a credible account of the processes in virtue of which data are stored, retrieved, forgotten, and so forth. Nor does the mathematics of the hologram appear to unlock the door to the mathematics of neural ensembles. The metaphor did, nonetheless, inspire research in parallel modeling of brain function. (See section 10.5.)
The dominant metaphor of our time likens the brain to a computer, though this dominance is perhaps owed less to tight-fitting similarities than to the computer's status as the Technological Marvel of our time. Only in a very abstract sense is the brain like a computer: in both the brain and the electronic machine the output is a function of the input and the internal processing of the input. But this is clearly a highly abstract similarity, drawing merely on the presumption of systematicity between input and output. Finding the relevant points of similarity such that knowing some fact about computers will teach us some principles of brain function is very difficult, and how helpful the computer metaphor really is remains an open question. Certainly there are profound dissimilarities between brains and standard serial electronic computers (see section 10.9), and it is arguable that for many brain functions the computer metaphor has been positively misleading. (See discussions by Von Neumann 1958 and Rosenblatt 1962.) Most pernicious perhaps is the suggestion that the nervous system is just the hardware and that what we really need to understand is its "cognitive software." The hardware-software distinction as applied to the brain is dualism in yet another disguise. In any case, which differences and similarities are trivial and which are significant cannot be determined independently of knowing something about how both brains and computers work. Metaphors can certainly catalyze theorizing, but theories they are not.
Second, flowcharts describing projection paths in vertebrate nervous systems are sometimes characterized as theories. Insofar as they are theories, they are typically theories of anatomical connections, sometimes with a highly schematic complement of physiological connections. Although they may suggest a rough description of what happens at each stage, they do not really explain the processes such that from a given kind of input, a given kind of output results. For example, the circuit diagram for the cerebellum is sometimes taught as though it were a theory of how the cerebellum coordinates movement, but in reality it is no such thing. (This example will be more fully discussed in the next section.) Circuit diagrams often represent a huge experimental investment, and they are absolutely essential in coming to understand the brain's functional properties, but they are not.
Third, sometimes a list of ingredients important for getting a theory is offered as the theory itself, but evidently such a list is not per se a theory of what processes intervene between input and output. A list may include items such as that the brain is in some sense self-organizing, that it is a massively parallel system, and that functions are not discretely localizable but in some sense distributed. But a list of this sort does not add up to a theory, though the items are relevant considerations to be stirred into the pot. Like the prohibition against homunculi, they might be construed as constraints that any serious theory will ultimately have to honor. Or in more old-fashioned language, they might be called "prolegomena for future theorizing." (See also below.)
Fourth, as Crick has said, it is important to know what problems to try to solve first, and what problems to leave aside as solvable later. Because one is ever on the brink of being thrown into a panic by the complexity of the nervous system, it is necessary at some point to put it all at arm's length and ask: What answers would make a whole lot of other cards fall? What are the fundamental things a nervous system must do? This of course will be a guess, but an educated guess, not a blind one. The hope of any theorist is that if the basic principles governing how nervous systems operate are discovered, then other operations can be understood as evolution's articulation and refinement of these basic principles. Simplifications, idealizations, and approximations, therefore, are unavoidable as part ~of the first stage of getting a theory off the ground, and the trick is to find the simplification that is the Rosetta stone, so to speak, for the rest. In physics, chemistry, genetics, and geology, simplifying models have permitted a clarity of analysis that lays the foundation for coping with the tumultuous complexity that exists. Accordingly, Ramon y Cajal's warning against ". . . the invincible attraction of theories which simplify and unify seductively" should not be taken too much to heart. If a theory is on the right track, then the initial simplifications will grow into more comprehensive articulations; otherwise, it will shrivel and die.
The guiding question in the search for theory is this: What sort of organization in neuron-like structures could produce the output in question, given the input? Different choices will be made concerning which output and input to focus on. For example, one might select motor control, visual perception, stereopsis, memory, or learning about spatial relations as the place to go in. What is appealing about visual perception is that we know a great deal about the psychology of perception and about the physiology of the retina, the lateral geniculate nuclei, and the visually responsive areas of the cortex. What we do not understand, amorig other things, is how to characterize the output at various anatomical stages. On the other hand, what is appealing about motor control is the inverse. We know what the output is - namely, motor behavior - and we know quite a lot about the structural layout of the cerebellum, the motor cortex, and other motor-relevant parts of the brain. But we do not understand how to give a functional characterization of the input to motor structures. Different theorizers, accordingly, will have different hunches about the best place to dig in.
In the most general terms, we are looking for a description of the processes intervening between the input and the output. Constraining the theory-construction will be facts at all levels of organization. Thus, if we are theorizing about how a visual representation is constructed from light patterns falling on the retina, we must bear in mind fine-grained facts (such as that the only light-sensitive elements are rods and cones), larger-grained neural facts (such as that there are numerous topographic maps on the cortex), and psychological facts (such as that color perception remains constant under varying conditions of illumination). In addition, there are facts about visual deficits under specified neurological insult. For example, monkeys with bilateral lesions to the inferior temporal cortex are selectively impaired in ' visual recognition tests, whereas monkeys with lesions to the posterior parietal cortex are selectively impaired in landmark discrimination (Mishkin, Ungerleider, and Macko 1983).
There are also real-time constraints. In other words, the time it actually takes the nervous system to accomplish something, together with the facts of conduction velocities and synaptic transmission times, will put an upper limit on what to hypothesize as the number of steps intervening between input and output. For example, if it takes about 500 msec for a person to respond in a visual recognition test, then there must be no more than about 100 synaptic steps between the input and the output. Accordingly, a hypothesis that envisages a serial processing unit for visual recognition with 300 or 1,000 steps cannot be right. This observation is usually followed by the inference that the brain, unlike the standard electronic computing device, is a massively parallel machine (Feldman and Ballard 1982, Brown 1984). The point is, 100 steps in a serial processing program is far too few to do anything very fancy. Certainly it is not remotely enough to do the sorts of superlatively complex things our brains routinely do. Considerations of real-time constraints have, accordingly, militated against the idea that the brain's mode of operation can be modeled by a sequential program.
In the remaining sections I shall offer a small sample of some of the kinds of theoretical ventures currently undertaken. Opinions diverge widely concerning what has promise and where the gold is. Generally speaking, theoretical approaches originating with neuroscientists are decried by those in the computer science business as "computationally naive"; on the other side of the coin, neurobiologists usually deplore the "neurobiological naivete" of those whose theories originate in computer science laboratories. So long as there is no theoretical approach known to do for neuroscience what Newton did for physics, we are all naive. Inevitably, there is a tendency to see one's own simplifications as "allowable provisionally" and someone else's as a fatal flaw. To one convinced of the gold in his own bailiwick, other theoretical diggings may seem crackpot. Additionally, if a theory has quite grand ambitions, it stands to be derided as "pie-in-the-sky"; if, on the other hand, a theory is narrow in scope and highly specific, it risks being labeled "uninteresting."
My approach here will be to present three quite different theoretical examples with a view to showing what virtues they have and why they are interesting. Each in its way is highly incomplete; of course each makes simplifications and waves its hands in many important places. Nevertheless, by looking at these approaches sympathetically, while remaining sensitive to their limitations, we may be able to see whether the central motivating ideas are powerful and useful and, most importantly, whether they are experimentally provocative. My strategy can be defended quite simply: if one adopts a sympathetic stance, one has a chance of learning something, but if one adopts a carping stance, one learns little and eventually sinks into despair.
Regardless of whether any of the three examples has succeeded in making a Grand Theoretical Breakthrough, each illustrates some important aspect of the problem of theory in neuroscience: for example, what a nonsentential account of representations might look like, how a massively parallel system might succeed in sensorimotor control, pattern recognition, or learning, how one might ascend beyond the level of the single cell to address the nature of cell assemblies, how co-evolutionary exchange between high-level and lower-level hypotheses can be productive. They all try to invent and perfect new concepts suitable to nervous system function, and they all have their sights set on explaining macro phenomena in terms of micro phenomena. Being selective means that I necessarily leave out much important work, but given the limitations of space, that is something I can only regret, not rectify.
Two of the examples originate from within an essentially neurobiological framework. The first focuses on the fundamental problem of sensorimotor control and offers a general framework for understanding the computational architecture of nervous systems. The authors of this approach are Andras Pellionisz and Rodolfo Llinas, and owing to the very broad scope and the general systematicity their theory seeks to encompass, I shall discuss it at considerable length. The second, originating with Francis Crick examines the neurobiological basis for certain attentional mechanisms specified by psychological hypotheses. This is more narrowly focused and can be discussed quite succinctly. The third approach, discussion of which I sandwich between these two, is a new development within the wider field of artificial intelligence research and goes by the name of connectionism or the modeling of parallel distributed processing (PDP). Connectionist researchers are trying to figure out the computational operations used in nervous systems, and the strategy has been to use computer models of parallel distributed systems to try to generate the appropriate macro phenomena from neuron-like elements in a network-like arrangement. In contrast to the other two, this approach is essentially based in computer science, but unlike standard artificial intelligence research, it is informed and constrained by neurobiology.
10.3 Tensor Network Theory
Because there are general philosophical lessons to be drawn concerning the possibility of a new "neurocognitive" paradigm and concerning the co-evolution of functional and structural hypotheses it will be useful to place the opening discussion of tensor network theory within the context of its inception-of what led to the first fumblings and how the general idea of phase spaces and coordinate transformations slowly took shape.
The place to start, then, is where the theory started: the cerebellum. With only some exaggeration it can be said that almost everything one would want to know about the micro-organization of the cerebellum is known. For neuroanatomists the cerebellum has been something of a dream of experimental approachability, because it has a limited number of neuron types (five, plus two incoming fibers), each one morphologically distinctive and each one positioned and connected in a characteristic and highly regimented manner (figure 2.4). The output of the cerebellar cortex is the exclusive job of just one type of cell, the Purkinje cell (of which more anon), and the input is supplied by just two, very different cell systems, the mossy fibers and the climbing fibers. This investigable organization has made it possible to determine the electrophysiological properties of each distinct class of neuron and to study in detail the nature of the Purkinje output relative to the mossy fiber-climbing fiber input. The neuronal population in the cerebellum is huge-something on the order of 10 to the 10th power neurons and there is at least another order of magnitude in synaptic connections. Nonetheless, basic structural knowledge of the cerebellum has made it possible to construct a schematic wiring diagram that illustrates the pathways and connectivity patterns of the participating cells (figure 10.1). The first point, then, is that a great deal is understood at the level of micro-organization.
Exactly what the cerebellum contributes to nervous system function is not well understood, however. What is known is that it has an important role: in coordinating movement, as well as in moving the whole body. It is what permits one to smoothly touch one's nose, catch an outfield fly, or land a snowball on a passing car. The complexity underlying any of these feats puts high demands on a nervous system. For example, in catching a fly ball, a baseball player must estimate the trajectory of the ball and keep his eyes on it while running to where it is expected to fall. So he has to run, visually track, maintain balance, reach to intercept, and finally catch the ball.
Subjects with cerebellar lesions show a decomposition of movement, almost as though the various parts of each movement had to be thought out one by one. Undershooting and overshooting the target and moving the limb in the wrong direction are also typical dysmetric signs in cerebellar subjects. Cerebellar patients also have difficulties in checking a fast movement, such as a swing of the arm. There are commonly problems in gait, showing themselves especially in unsteadiness and large stride. Depending on the area of lesion, there may also be motor impairment of speech (dysarthria). Playing baseball is out of the question.
It is known that the cerebellum is not essential for movement because subjects with a nonfunctioning cerebellum can still make voluntary movements. But evidently it is essential for well-controlled, well-timed, well-spaced movement. Plasticity in the nervous system does permit some compensation in the event of cerebellar lesions occurring early in development. Children whose cerebellar hemispheres are damaged early in life may nonetheless develop quite good motor control, so long as the more medial structures in the cerebellum (the flocculonodular lobe and the vermis) are undamaged. But if these structures are also damaged and the entire cerebellum is nonfunctional, the child remains ataxic (that is, suffers deficits in motor coordination) and dysmetric.
The evolution in complexity and size of the cerebellum in humans is at least as striking as that of the cerebrum. Correcting for body size, humans have a larger cerebellum than, for example, chimpanzees, whose cerebellum in turn is larger than that of horses or dogs. As one might predict, therefore, human versatility in motor control is remarkable. To mention only a tiny sample, we can swim, pole-vault, climb trees, use knives, speak languages, whistle, draw, skate, and play musical instruments. For each of these accomplishments the nervous coordination of muscles is a stunningly complex affair.
Circuit diagram for the cerebellar cortex. Purkinje cells are excited directly by climbing fibers and indirectly (via parallel fibers from the granule cells) by the mossy fibers. Stellate and basket cells, which are excited by parallel fibers, act as inhibitory interneurons. The Golgi cells act on the granule cells with feedback inhibition (when excited by parallel fibers) and feedforward inhibition (when excited by climbing and mossy fiber collaterals). The output of the Purkinje cell is inhibitory upon the cells of the intracerebellar and vestibular nuclei. (Modified from Ghez and Fahn (1981). Ch. 30 of Principles of Neural Science, ed. E. R. Kandel and J. H. Schwartz, pp. 334-346. Copyright 1981 © by Elsevier Science Publishing Co., Inc.
_______
What is the input on which the cerebellum can work its miracles? It includes massive inputs from the cerebral cortex-the motor strip and nearly everywhere else-as well as from other brain regions subserving motor function. The cerebellum also receives afferent inputs from all types of sensory receptors. Some of the cortical inputs are thought to be grossly specified motor commands for which the cerebellum provides the finely tuned, detailed commands. (This will be elaborated below.) In the absence of the cerebellum the motor commands of the cerebral cortex are conveyed down the spinal cord without the coordinative tuning of the cerebellum.
Now if we know so much, in a general fashion, about what the cerebellum does, and if we know so much about the fine-grained structural facts, we ought to be able to figure out how the cerebellum does what it does. We ought, that is, to be able to explain how the activity of the collections of cells produces coordinated movement. For anyone who hoped that the theory would simply tumble out once so many details were available, the cerebellum seemed strangely frustrating. Because what remained mysterious was the functional story-intermediate between the gross functional description and the wiring diagram-that would explain exactly what role the cerebellum plays in the administration of motor control. The epistemological situation provoked diverse researchers into trying to find a fruitful theoretical orientation (for example, Braitenberg and Onesto 1961, Marr 1969, Ito 1970).
The line that Pellionisz and Llinas pursued depended on their determination to take as the starting point the parallel nature of information processing in the brain, and in the cerebellum in particular. If the cerebellum has a parallel architecture, in the sense that many channels are simultaneously processing information, then, they argued, it is a fair assumption that the computational processes are suited to that architecture. To understand what the computational processes might be, they followed the idea that they needed to know about the patterns of activity within large arrays of neurons.
A "wiring diagram" of cerebellar neurons is useful in describing in a highly schematic way the connections between input and output. Typically, however, the diagram displays one or two schematic neurons and their connections, whereas in fact these are embedded in an array of thousands of cells. That is, the massively parallel nature of the network is, for graphic convenience, suppressed. Such suppression will not matter if the schematic neuron is a faithful representative of every neuron in its array-if, that is, the system is essentially redundant.' On the other hand, if the global connectivity pattern within the array is itself crucial to how the array processes information, then we pay for the convenience of the suppression, inasmuch as we mask exactly the detail we need in order to understand the system.
Now in fact neurons in an array do appear to differ in number of synaptic connections with a given incoming neuron (convergence), number of neurons to which they project (divergence), synaptic morphology, and so forth. For example, and this example will be important in the tensor network theory, sets of Purkinje cells positioned at different sites along a beam of incoming parallel fibers have different outputs, and the differences are systematic (Pellionisz, Llinas, and Perkel 1977). At least in this case the schematic neuron is not a faithful representative of all neurons in its array, and the differences, argued Pellionisz and Llinas, are not trivial but essential to the nature of the array's output. As they saw it, to understand those differences is to get close to understanding the functional story implemented in the parallel architecture.
Given this starting point, the task was to find out more about the patterns of activities between neuronal arrays. Because of the technological difficulties involved in simultaneous intracellular recording from multiple adjacent cells, Pellionisz and Llinas began instead by modeling a segment of a frog cerebellum in a computer in order to force more pattern into the open (Pellionisz, Llinas, and Perkel 1977). By drawing on the available knowledge of cell connectivity and morphology, they programmed a computer to "grow" huge numbers of cells (8,285 Purkinje cells, 1.68 million granule cells, 16,820 mossy fibers), with the appropriate connectivity network, thereby creating a fictive cerebellum in the computer to simulate the real thing. They could then activate specific input cell populations and investigate the patterns of activity in large populations of receiving cells.
The model is, of course, just a model, limited by whatever anatomical and physiological data are built into it. Therefore, no grand and incontestable conclusions about how the cerebellum works should be drawn directly from it. Nevertheless, the model is a valuable heuristic device because it enables us to see something not visible through single-cell recordings-namely, patterns of activity in huge (fictive) neuronal ensembles. It enables the theoretical imagination to transcend the limits of the schematic wiring diagram and single-cell recordings and to begin to come to terms with the parallel nature of the system. So even if no computational conclusions can be drawn, testable computational hypotheses may germinate.
Once convinced that the connectivity of arrays of neurons is crucial to explaining how a given input yields a given output, the investigator must find a way to characterize the relation between input arrays and output arrays. In mulling over the patterns the computer simulation yielded and the problems the cerebellum had to be solving as its contribution to sensorimotor control, Pellionisz and Llinas began to think that what the network of cerebellar cells did to its input could be characterized by means of a tensor-a generalized mathematical function for transforming a vector into another vector, no matter what the frames of reference involved. The basic mathematical insight was that if the input is construed as a vector in one coordinate system, and if the output is construed as a vector in a different coordinate system, then a tensor is what effects the mapping or transformation from one vector to the other. Which tensor matrix governs the transformation for a given pair of input-output ensembles is an empirical matter determined by the requirements of the reference frames in question. And that matrix is implemented in the connectivity relations obtaining between input arrays and output arrays.
Let us consider this step by step. Vectors are represented geometrically as directed line segments in a specified coordinate system (frame of reference). The various components of the vector are given in terms of their values as specified in relation to the relevant coordinate axes (figure 10.2). If each neuron in a network of input neurons specifies an axis of a coordinate system, then the input of an individual neuron-its spiking frequency-defines a point on the axis, and the input of the whole array of neurons can then be very neatly given as a vector in that space. Similarly, the output of an array can be specified as a vector in the space defined by the set of output neurons. (For an introduction to the basic concepts, see Jordan 1986).
Given the data on input vectors and output vectors supplied in the model, Pellionisz observed that from a mathematical standpoint, the connectivity relations between input and output neurons serve as a matrix, such that any input vector is transformed into an output vector. That is, the nature of the regularity in the patterns of activity of the neuronal arrays represented in the model invited the hypothesis that the arrays are doing matrix multiplication. In particular, the systematic differences in response profiles of sets of Purkinje cells situated at different locations on the same beam of parallel input fibers could be explained as the outcome of matrix multiplication (figure 10.2).
Consider a simple case of a 2 x 3 matrix - that is, two rows and three columns, as illustrated. Let the vector be (3,2). To find the dot product, multiply the first component of the vector by the matrix number in row one, column one; multiply the second component by the matrix number in row two, column one. Add the two products to yield the first component of the resultant vector. Repeat for columns two and three to find the second and third components.
_________
So far the vector-matrix mathematics seems like a marvelously convenient way to order a lot of messy, fine-grained detail, but suppose that the cerebellum's susceptibility to a vector-matrix analysis reflects a deeper functional reality. With this thought in mind, Pellionisz began to pursue a further hunch: suppose that ensemble activity can be described as mapping vectors onto vectors not as a matter of mere mathematical convenience but because ensembles really represent coordinate systems, and a fundamental functional problem for a nervous system consists in making translations from one coordinate system to another. What coordinate systems? Well, those defined by the representational job of a given ensemble.
To begin with, there will be the coordinate system specified by visual or olfactory or vestibular input arrays and the very different coordinate system specified by motor output arrays. Suppose, indeed, that the fundamental computational problem of sensorimotor control is the geometrical problem of going from one coordinate system (e.g., visual) to another, very different coordinate system (e.g., motor). Then arrays of neurons are interpretable as executing vector-to-vector transformations because that is what they really are doing the computational problems a nervous system has to solve are fundamentally geometrical problems. The idea seemed to have plausibility not only for the cerebellum but for wider domains as well. [1]
A tensor is a generalized mathematical function for transforming vectors into other vectors, irrespective of the differences in metric and dimension of the coordinate systems. If the basic functional problem of sensorimotor control is getting from one very different coordinate system to another, then tensorial transformations are just what the nervous system should be doing. Accordingly, the hypothesis is that the connectivity relations between a given input ensemble and its output ensemble are the physical embodiment of a tensor.
The geometric characterization of the problem of sensorimotor control, and of neurofunctional capacities generally, is neither immediately compelling nor, for that matter, immediately comprehensible. What is required is something on the order of a major conceptual shift. The phenomenological scenario here seems to be confusion and incomprehension in the first phase, followed, as understanding flowers, by a gathering sense of obviousness adhering to the general principles. The detailed hypotheses are, evidently, a further matter. My own understanding here began to find its feet as Paul M. Churchland and I constructed a cartoon story of a highly simplified creature who faces a sensorimotor control problem of the utmost simplicity.2 In what follows I shall use the cartoon story in trying to outline the Pellionisz-Llinas picture of the brain's geometrical problems and its geometrical solutions. With that in hand, we shall return to nervous systems and to the cerebellum in particular. First, however, a brief philosophical aside.
For purists of the top-down persuasion, the cardinal article of faith is that first you figure out what the mind-brain does, and secondarily you find out how it might implement the functions described. Granted, in a certain sense, any theorizing about mind-brain function has a veneer of top-downishness, else it would not be theorizing but data-gathering. If the dominant connotation of "top-down" is that of the purists, however, then to the degree that the theorizing is highly constrained and richly informed by implementation-level data, it is decidedly confounding to label the enterprise as "top-down."
In the case of tensor network theory the insights concerning the functional nature of sensorimotor control grew out of reflections on the significance of vector-matrix descriptions at the level of cell assemblies, which were themselves enabled by computer simulations dependent on a massive data bank of structural detail. In short, the high-level functional hypothesis was suggested by the low-level functional hypothesis, which in turn was a consequence of adopting a strategy based on essentially structural considerations. This is exactly the reverse of the order of discovery advised by the top-down purist. Poetic distortion aside, it is tempting to see the conceptual genesis of the tensor network theory as an instance of figuring out how something works before figuring out what it is doing.
I do not wish to make excessively much of this point, and it by no means entails anything terribly grand, such as that there is no distinction between functional capacities and their structural implementation. Nevertheless, I do see it as enfeebling the methodological advice of the top-down purists, as well as bolstering the stock of the co-evolutionary approach to cognitive neurobiology.
10.4 Cartoon Story of What a Tensor Does in Sensorimotor Control
The cartoon world is inhabited by a very simple crab-like critter, Roger. He is equipped with a pair of eye-like devices for detecting the position of an apple in external space, where each eye can rotate in a socket so as to get the apple in register with its "sweet spot" (fovea, as it were). The eyes can rotate ninety degrees either to the right or to the left of their straight ahead position. Roger also has an arm-like device, a two-jointed limb that consists of a forearm and an upper arm, the latter projecting from midway in the center structure. The arm is used to make contact with the apple (figure 10.3). Although conceptually Roger is a three-dimensional critter, his existence as a computer-generated display means he is limited to activity in just two dimensions, as though he never pays any attention to height. His world, to make things simple, is really just a 2-D, flatlander world. Figure 10.3 (part b) shows this world viewed from above.
The apple has a position in external space, what we shall call 2-D external Euclidean space. The position of the apple in this external space can be given by drawing a pair of coordinate axes and specifying the position in terms of the coordinates. Its position can also be represented in visual phase space-that is, its position in the natural coordinate system of Roger's sensory equipment. How do we characterize Roger's visual phase space? Since each eye can rotate, the position of each eye can be specified by the angle it subtends as it turns away from the straight ahead position (see figure 10.3). It is most convenient to characterize the straight ahead position as 90; hence, all eye positions can be specified as the angle subtended by the horizontal axis and the "fovea line." For example, suppose that the apple is at (1.2, 10.8) in external space. Then to foveate it, the eyes must rotate: the left, such that the angle subtended by the horizontal and the "fovea line" is 65 degrees; the right, such that the relevant angle is 105 degrees. In Roger's visual phase space, therefore, the position of the apple is given by the ordered pair of position angles for each eye, namely (65, 105).
The problem of sensorimotor coordination. (a) and (b) depict a crab-like robot with rotatable eyes and an extendable arm. As the eyes triangulate a target by assuming angles (alpha, beta), the arm joints must assume angles (theta, phi) such that the tip of the forearm makes contact with the target. (Adapted from Paul M. Churchland (forthcoming).)
___________
Roger's visual space is a phase space in which the position of the apple is represented by the two eye-rotation angles, alpha and beta, that jointly triangulate it. Any coordinate system specifies a phase space, and here "position in phase space" simply refers to the global condition of the physical system being represented. Phase spaces may differ as a function of the number of coordinate axes (2, 3, 50, 10,000, etc.) and the angles of their axes (at right angles to each other, or non-orthogonal). A hyperspace is a phase space with more than three dimensions. A phase space may be Euclidean, but it need not be. It could, for example, be Riemannian, in which case the interior angles of a triangle inscribed in. that space need not sum to 180 degrees. (A caution: the term "phase space" is commonly used in classical mechanics to denote a specific coordinate space of six dimensions, three for position and three for momentum. But as I use the term here, it has the entirely general meaning of "coordinate space" or "state space." See also Suppe 1977.)
Note that for any position of the apple in external space, there is a corresponding position of the apple in Roger's visual phase space. Accordingly, we can say that Roger s visual vector, such as (65, 105), represents the position of the apple in the world, since there is a systematic relation between where the apple is in the world, as described in external coordinates, and "where" in visual phase space it is, as specified by a pair of eye-angle coordinates (figure 10.4a).
Just as Roger has a 2-D visual space in which the position of the object is represented, so he has a 2-D motor space in which its arm position can be represented. But, and this is crucial, these two phase spaces are very different. How do we characterize Roger's motor phase space? Again, by specifying the axes appropriate to his motor equipment. This time, the position of his limb in phase space is given by the two angles by which it deviates from a standard position. Thus, let the zero position of the upper arm be flush with the horizontal axis. Then a position of 45 on the upper-arm axis will represent an upper-arm position of 45 degrees off the horizontal (figure 10.3). Correlatively, let the zero position of the forearm be its position when extended straight out from the upper arm, wherever the upper arm happens to be positioned. Accordingly, a position of 78.5 on the forearm axis represents the forearm as rotated 78.5 degrees counterclockwise from the line extending out from the upper arm, whatever the position of the upper arm. Notice, therefore, that we can give the overall position of Roger's arm in motor phase in terms of the two angles as (45, 78.5). Moreover, we can specify the position of the apple by specifying that arm position where the tip of the forearm just touches the apple. The sensory phase space and the motor phase space are represented in Roger's "brain" and reflect the unique nature of the sensory apparatus and the motor apparatus, respectively (figure 10.4).
The respective configurations of the crab's sensory and motor systems can each be represented by an appropriate point in a corresponding phase space: (a), (b). The crab needs a function from points in sensory phase space to points in motor phase space. (Adapted from Paul M. Churchland (forthcoming).)
___________
Now for the action (figure 10.5). The eyes, having detected the apple, announce its position: "Apple at (55, 85)." Notice that if the arm were to take as its command, "Go to (55, 85)," then it would execute that command by putting the upper arm at 55 and the forearm at 85. This would be disastrous, because it would put the arm nowhere near the apple. In other words, the position of the apple in visual space is not the position of the apple in motor space. From figure 10.5 (part a) it is evident that if the apple is at (55, 85) in visual space, it is at (62, 15) in motor space. What Roger needs, therefore, is something to tell him what coordinates in motor space correspond to a given set of coordinates in visual space. If Roger s arm is to go where the apple is, he needs to know its "apple-touching" position, and figure 10.5 shows the path in motor phase space his hand should follow from its starting position folded up against his chest. That is, he needs something that will tell his limbs where, to go in motor space on the basis of where the apple is in visual space. He needs a mathematical function that will specify where in motor space to go, given the location of the target in visual space-in other words, something to effect a transformation of coordinates. Figure 10.5 shows three other movements the arm makes (b, c, d), depending on where the target is placed. In general, the mathematical function will compute the target's position in motor space on the basis of its detected position in sensory space.
The successfully coordinated crab. In this computer simulation the sensory phase space position is entered as input, and the motor phase space position is computed as output. The arm configuration is then directed along a straight line in motor phase space, from its folded rest position (0, 180), toward its target position. See upper right inset in each example. That position places the arm in physical contact with the target object in real space. (Adapted from Paul M. Churchland (Forthcoming).)
Let us consider the desired mathematical function in pictorial terms. Figure 10.6 (part a) displays Roger's sensory or eye-position phase space, and the visible grid lines represent that portion of the phase space such that Roger's eyes converge on a point within reach of Roger's modest arm. Figure 10.6 (part b) displays Roger's motor phase space, but with a grid of curving lines superimposed on it. Their significance is as follows. For each point in sensory phase space, there is a uniquely corresponding point in motor phase space, a point that specifies Roger's arm as touching the triangulated target. If we now consider an entire grid of points in sensory phase space, there will of course be a corresponding "grid" of positions specified in motor phase space. That is what we see in figure 10.6. But as projected onto motor phase space, that "grid" is deformed relative to the original sensory grid. (The triangle and rectangle are added to help the reader locate the original positions within the deformed grid.) Accordingly, we may think of the coordinate-transforming function at issue as a transformation that deforms sensory phase space in order to put all the points in it into proper register with the desired points in motor phase space. "Proper register" here just means that Roger's arm systematically reaches out to wherever his eyes triangulate.
Exactly what transformation is required is clearly very sensitive to the details of Roger's sensory and motor equipment. If his eyes were farther apart, or if his arm segments were of different lengths, a somewhat different transformation would be in order.
The coordinate transformation, graphically represented. The grid in (a) represents the set of points in sensory phase space that correspond to a triangulated object within reach of the crab's arm. For each such point in (a), its corresponding position in motor phase space is entered in (b). The entire set of corresponding points in (b) displays the global transformation of phase-space coordinates effected by the crab's coordinating function. The heavily scored triangle and rectangle illustrate corresponding positions in each space. (Adapted from Paul M. Churchland (forthcoming).)
__________
The critical point, therefore, is that we need a way of going from points or vectors in sensory space to points or vectors in motor space. Clearly there must be a systematic relationship between the "sensory position" and the "motor position" of the apple, or else poor Roger would never manage to grasp what he senses. A mapping from vectors in one space to vectors in another space can be usefully represented as a general transformation of the coordinates of the one space into the coordinates of the other. For spaces in general (including non-Cartesian spaces) the coordinate transformer is called a tensor. In real animals, Pellionisz and Llinas argue, the intrinsic geometry of nervous subsystems need not be limited to phase spaces with orthogonal (Cartesian) axes, and thus the relevance of tensors. But the point of our cartoon story is merely to illustrate the principle of coordinate transformation, and so we shall pass by the complexities of the fully general case.
In this cartoon story the position of the apple in real space is the invariant, which the visual system and the motor system both represent, each in its different way, while the coordinate transformer tells the effector system what it must do to make contact with the invariant. In geometrical terms, the coordinate transformation tells us how we have to deform one phase space to get at the object in the other phase space. Tensors are a means whereby the nervous system can represent the very same thing many times over, despite the differences in coordinate systems in which the thing is represented. In sum, then, representations are positions in phase spaces, and computations are coordinate transformations between phase spaces.
Bear in mind, however, that even equipped with a coordinate transformer to translate sensory locations into the correct motor locations, Roger is radically simplified-so much so that he would not stand a chance in the real, cutthroat, biological world. Even the humblest nervous systems are more complex and more sophisticated than Roger's. To begin with, his is a 2-D world, and ours is a 4-D space-time world. As soon as we consider the sensorimotor control problems that must be solved by the brains of real creatures, making their living in a 4-D space-time world, the necessity of elaborating on the simple coordinate transformer of the cartoon story is plain. Moreover, Roger never moves his whole body, and he never has the problem of maintaining posture and balance. All he does is visually locate the apple and then touch it. He does not flee or hide from any predator, he does not mate, he does not build a nest or dig a hole, he does not even chew up and swallow the apple he touches. Consider also that although Roger has merely two phase spaces-one afferent and one efferent-this is an unrealistic arrangement for a real nervous system, which could be expected to have some number of intervening phase spaces as well. Nevertheless; the crux of the Pellionisz-Llinas approach is this: the sensorimotor problems faced by more realistic creatures can be understood as reducing at bottom to the same general type of problem that Roger faces-namely, the problem of making coordinate transformations between different phase spaces. And the solution found for Roger's cartoon world illustrates the general nature of the solution evolved by organic brains in the real world.
Roger is simplified in a further dimension. He has neither muscles nor neurons to make muscles contract. When he moves an arm, that is really just the computer painting lines across the CRT screen. When a real crab moves a claw, it does so by virtue of the precise orchestration of muscle contraction by neurons. Let us wallow a bit in the sensorimotor predicament of a real crab foraging for food. Supposing it spots an edible chunk of fish, it must move toward it, grasp it with its claw, and get it into its mouth. It has to contend with six legs; moreover, each leg has three joints, each joint is served by at least two muscles, and each muscle consists of many muscle cells and is innervated by a large number of neurons. If the object to be intercepted is itself moving, the control problem becomes very complex. But it is approachable by using the same basic mathematical idea used in solving Roger's problem.
If we can think of the crab's arm as specifying a phase space, then the set of muscles concerned may also be thought of as specifying a phase space, where the positions of each muscle are represented on a proprietary axis of that space, and where a vector in that space is determined by the degree of contraction of the component muscles. A phase space of yet higher dimensions is specified by the motor neurons innervating the muscles, where each neuron is given a dimension and its firing frequency will be represented as a point along that axis. Notice that we can expect there to be systematic relationships between positions in the skeletal phase space, positions in the muscle phase space, and positions in the neuronal phase space. The central idea is quite simple: the limbs move the way they do because the muscles contract the way they do, where that pattern of effects is in turn caused by a pattern of neuronal activation of the muscle units.
Animals' motor systems had to evolve systematic relationships among the phase spaces of motor neurons, muscle cells, and limbs if they were going to use neurons to control the movement of muscles and thereby control the movement of limbs. Any animal whose motor system lacks such relationships will not be able to move properly, and its survival time will be brief. From the perspective of tensor network theory, to look for the functional relations between connected cell assemblies is to investigate the properties of the relevant phase spaces-that is, to determine their geometries, and to determine the transformations that will take us from the representation of some external invariant in a given space to its representation in a different phase space. Knowing the geometry of the limb phase space, therefore, will guide us in approaching the motor neuron phase space.
Similar points apply of course to matters on the afferent end. Roger does not detect the presence of the target by virtue of photosensitive neurons. Biological organisms with real eyes do. Nevertheless, the basic principle of representation as position in phase space and computation as coordinate transformation can be invoked. That is, in real organisms retinal neurons will specify a phase space, vestibular neurons will specify a phase space, and so forth. If the afferent system is to play a role in the organism's feeding, fleeing, and so forth, then afferent phase spaces will have to be coordinated with efferent phase spaces. Sensory phase spaces are bound to be different from motor phase spaces, since, to put it schematically, the first must be an "as-the-world-presents-itself" representation, whereas the second must be an "as-my-body-should-be" representation. If nervous systems are to represent an invariant, as they must do if animals are to intercept prey, then tensors appear to be an efficient way in which the sensory representations can be transformed into output representations in motor phase spaces. And we can envision the evolution of fancier sensorimotor control aided by the development of phase spaces intervening between afferent and efferent. If a tensor equation is valid in one frame of reference, it is valid in all, regardless of how deformed one space is relative to another or whether the spaces differ in dimensions. On this view, the specific connectivity of distinct neuronal arrays has evolved to effect these general tensorial transformations under the specific conditions of a given species of organism.
In living organisms, then, it is arrays of neurons that must represent positions in phase spaces such as visual space or motor space, and it is a neuronal network that must make coordinate transformations. To see how a neuronal network can be suited to the implementation of coordinate transformations, let us start with a simple schematic vector-to-vector transformation. Consider an input system of four dimensions whose inputs a, b, c, d are transformed into output values x, y, z, of a three-dimensional system. The input can also be regarded as a point in the 4-D phase space, or as a vector whose base is at the origin of the relevant phase space and whose tip is specified by the four components of the input. Similarly, the output can be regarded as a point in 3-D phase space, or as a vector whose tip is determined by the output values.
Suppose the input vector is transformed into the output vector by matrix multiplication. This mathematical operation can be realized rather simply by the schematic neural array in figure 10.7. The array consists of three main structures: the parallel input fibers, the dendritic tree of the receiving cells, and the axons carrying the output of these cells x, y, and z. The parallel fibers carry excitatory input in the form of action potentials, and the values a, b, c, d are determined by the momentary spiking frequency of each of the four fibers. Every parallel fiber makes synaptic contact with each of the three dendritic trees. The output frequency of spike emissions for each neuron is determined by (1) the frequency of input stimulations it receives from all incoming signals and (2) the nature of the synaptic connectivity of each input junction, where this includes such factors as distance from the axon hillock, size of receptor site, and so forth. The latter values are represented by the numbers in the matrix in figure 10.7. The neural connectivity, therefore, models the matrix. The signals are "summed" at each axon hillock and a spike train is emitted. Thus, the output vector has as its components x, y, and z, the three output frequencies. Vectors (input) are thus transformed into vectors (output) via matrix multiplication.
Coordinate transformation by matrix multiplication, neurally implemented. The input vector (a, b, c, d) is physically represented by four spiking frequencies, each of which is above (positive number) or below (negative number) the baseline spiking frequency of the input pathway. Each input element synapses onto all of the output cells, and the weight of each synaptic connection implements the corresponding coefficient of the abstract matrix. Each cell "sums" its incoming stimulations and emits spikes down its output axon with a frequency proportionate to its summed input. Thus results the output vector (x, y, z). (Adapted from Paul M. Churchland (forthcoming).)
_________
Although the model neuron array is highly schematic, it is fairly easy to imagine how to embellish it. For example, the dimensionality of the phase spaces can be increased by adding neurons, redundancy can be accommodated if needed by "twinning" neuron configurations, the matrix can be made plastic by allowing for modifiability of synaptic junctions or for the addition of receptor sites. Moreover, something resembling this neuronal array does exist in the cerebellar cortex. Indeed, it was precisely by pondering the regimented organization of parallel fibers and Purkinje cells in the cerebellar cortex that Pellionisz and Llinas came to the view that their basic principle of operation was vector-to-vector transformation. (See again figure 10.2.) The set of incoming parallel fibers specifies a vector, the connectivity interface of parallel fibers and Purkinje dendrites models a matrix, and the axons of the Purkinje cells specify the suitably transformed output vector (figure 10.8).
There are in the cerebellum other matters to be factored in, such as the role of incoming climbing fibers (one to each Purkinje cell with multiple synaptic contacts) and the function of neurons in the cerebellar nucleus. But Pellionisz and Llinas consider that these can be accommodated within the basic framework of phase space representation and vector-to-vector transformation (Pellionisz and Llinas 1982). If the cerebellum is executing tensorial transformations, the next question is this: what is the character of those phase spaces such that a vector from one is transformed into a vector of the other? The answer to that depends on the empirical facts about what the motor cortex and the cerebellum are really up to, but at least a rough answer, based on clinical, physiological, and anatomical data, is already discernible.
Crudely, the plot line is this: the input from the cerebral cortex specifies in a general way what bit of behavior is called for. For example, suppose the incoming "intention" to my cerebellum is "Touch that (apple) with my right hand." The incoming "intention" vector specifies this position in a sensorimotor coordinate system (touch that seen/heard object), but it does not specify a curve in the motor space that says exactly how the goal position is to be achieved such that the target is intercepted. It is the job of the cerebellum to transform that intention vector into an execution vector that will orchestrate the motor neurons in order to produce a precisely and smoothly coordinated sequence of muscular contractions. It will have to coordinate all the muscles relevant to the behavior, based on an updated representation of the body's current configuration. No matter what my body's starting position-arms hanging straight down, arms over the head, arms behind the back, fingers in ipsilateral ears, fingers in contralateral ears, fingers between the toes-I can still touch my nose, and touch it in one smooth, graceful, fast movement. Nor is feedback necessary, except at the tail end of the movement as the finger closes in on the target, and then, notice, the finger decelerates. Often a movement is too fast to exploit feedback, as for example in the case of the finger movements of an accomplished violinist, or in catching an egg after it slips from one's hand. In such cases the conduction velocities of neurons are too slow to permit feedback data to be used to inform the next motor command, and the movement must be composed as a unified sequence without waiting for feedback. The coordinate transformation idea explains how this can be done.
Schematic diagram of the cerebellum acting as a space-time metric tensor of the motor hyperspace. The diagram represents the cerebellar input as a covariant vector, and the cerebellar network transforms it into a contravariant output vector. Abbreviations: MF, mossy fibers; GC, granule cells; PF, parallel fibers; PC, Purkinje cells; CN, cerebellar nuclei; BN, brain stem nuclei. The i[sub]k(t) motor intention components refer to time-point f; the e[power]n(T) motor execution components refer to time-point T, where T = t + d[power]n. The matrix elements in the array between Purkinje cells and cerebellar nuclei show the coefficients by which the mossy fiber information must be multiplied to yield the components of the execution vector expressed in summed firing frequencies of Purkinje axons arriving at cerebellar nuclei. e[power]n = (105, 22, -20)T. The Purkinje cell arrangement along a parallel fiber beam represents a "temporal lookahead module," implying that some supernumerary Purkinje cells take first- or second-order time derivatives of the input. (From Pellionisz 1984.)
_______
Further evidence for the composition of motor sequences in the absence of feedback is available from both animal and clinical studies. Deafferentation of a body part means that all afferent neurons from that body part are rendered unable to transmit their input to the CNS.
This includes not only skin afferents but also muscle and joint afferents. In experiments on monkeys it has been found that the animal can make good use of deafferented limbs to reach, grasp, climb, and walk; moreover, it can learn new movements (Taub 1976). In a rare case of deafferentation in a human, caused by a peripheral sensory neuropathy in which the motor neurons were spared, the patient remained able to perform many motor skills. For example, without visual feedback he could easily touch his nose, touch each finger sequentially with the thumb, and draw (on command) circles, squares, and figure eights. Evidently in these instances he was executing the sequence of movements without benefit of any feedback at all. Remarkably, this patient was even able to drive a car with a gear shift, though when he bought a new car he could not learn to drive it and had to resort to driving the old car (Marsden, Rothwell, and Day 1984). Given the touchy and idiosyncratic nature of clutches, this is not surprising.
Now the general principle proposed for cerebellar motor coordination is that the incoming "intention" vector is transformed, via a tensor, into an execution vector that specifies the detailed sequencing of muscle cell activity. Failure of the cerebellum to function means that the "intention" vector rather than the "execution" vector is the motor command directly transmitted down the spinal cord, and the result is inadequate muscular coordination and inappropriate timing. The finger overshoots or undershoots, and the movement lacks grace and smoothness, not unlike Roger's fumble if his visual space coordinates are directly used to specify his arm position in intercepting the target. Of course, the cerebellum may be doing other things as well, and it is also likely that there is not one massive connectivity matrix for motor coordination, but rather sets and even hierarchies of matrices. Nevertheless, the hypothesis invites us to see tensorial transformation as a fundamental principle of operation.
In sum, the tensor network hypothesis says that a neuronal network implements its general function as a connectivity matrix to transform input vectors into output vectors. There are, accordingly, two important strands in the hypothesis: the first accommodates the fact that the coordinate systems of neuronal ensembles will specify different frames of reference but must be systematically related, and the second accommodates the parallel nature of neuronal networks, by proposing that individual neurons in the array contribute the components to the vectors, while the structure of the connectivity between neuronal arrays determines the tensorial matrix. It is by trying to do justice to the parallel nature of nervous systems that one comes to fathom how they could use tensorial transformations to achieve sensorimotor control.
The idea that the tensor network approach really provides a theoretical framework within which questions about brain function can be addressed and answered is still very new, and not surprisingly the assessment within neuroscience varies. Some neuroscientists are suspicious that it comes from the "in-a-single-bound-Jack-was-free" school of thought. Some are uncertain about what it all comes to experimentally and whether this is really what a large-scale theoretical approach should look like. Others are enthusiastic because they have begun to envision how they can apply it in their own experimental research and because it literally gives them interpretive hypotheses for their data.
Naturally, it is possible that the tensor network theory is, after all, merely a dead-ender, despite the conviction of Pellionisz and Llinas that it is the real McCoy, or at least a robust and fertile ancestor of a real McCoy. In beginning to determine this, what essentially matters is whether the tensor network theory makes a difference in explaining and predicting experimental results. Quite simply, to be taken seriously it must engage the data: it must unify results, it must give coherent explanations, it must be testable, and it must open experimental doors.
We have already seen a crude analysis of its explanatory capacity with respect to the motor coordination function of the cerebellum, but in order to get a better look at the explanatory potential of the Pellionisz-Llinas approach, it will be useful to focus more closely on other domains where it appears to yield results. Let us therefore leave Roger in his simple, timeless, flatlander world and return to the neuronosphere.
10.5 Tensor Network Theory and the Vestibulo-Ocular Reflex
Gaze control is a rather complex affair involving many elements, including image-slip on the retina, contraction of the neck muscles, contraction of the muscles controlling eyeball movement, and motion of fluid in the vestibular apparatus of the inner ear. The vestibulo-ocular reflex (hereafter, the VOR) contributes one dimension to gaze control, and although in reality it is not isolated from other aspects, to begin by treating it as isolated is a useful idealization.
The VOR is the neuronal arrangement whereby a creature can continue to look at an object even though the head moves in any of its possible directions (all directions, if the creature is an owl). Rotation of the eyes is produced to compensate precisely for the movement of the head. As the head rotates to the right, the eyes track the object by rotating to the left. Here is a simple way to show yourself how clever the VOR is. First, stretch a hand in front of your eyes and, while holding your head steady, wave the hand back and forth quite quickly. Visually track the hand, and try to keep a steady, clear visual image. What you will get instead is a smeary image. For the contrast, keep the hand steady, and move your head back and forth at a good clip. Now the image of the hand is not smeary, as before, but clear and quite steady. That effect is owed to the VOR.
How does the VOR work? First we need to know what neuronal structures are involved. Principal elements are as follows: First, there are the semicircular canals of the vestibular apparatus in the ear, three canals on each side, one in each of the three planes. Although commonly thought to be precisely at right angles to each other, the canals in some species deviate considerably from the presumed ideal. Second, each eyeball has six extraocular muscles (muscles attached to the eyeball exterior) for rotating the eyeball in its socket. Basically, the VOR system circuit is at least a three-stage affair connecting vestibular receptors, via three stages of neuronal links, to the eyeball muscles. These stages consist of (1) the neurons responding to input signals from vestibular receptor cells, which synapse on (2) neurons in the vestibular nuclei (called secondary vestibular neurons), which synapse on (3) oculomotor neurons that innervate the muscles (figure 3.9). Notice that although only one neuron is sketched in to depict each stage, in reality of course there is a large array of neurons at each stage.
The problem for the system to solve is how much each muscle unit should contract in order that the eyeball move to compensate for the movement of the head. If we conceive the problem in terms of the tensor approach, it takes this form: Assume the system wants to keep a particular object in view while the head turns. Then the system needs to convert a new "head position" vector into a new "muscle position" vector. The input space will have three dimensions, one for each canal, and the relevant point along the axis for each canal is determined by the angle from the "initial" position (figure 10.9). As the head deviates from the initial position, we can describe its movement as a sequence of points in this vestibular phase space. For that sequence of points, we must find the "compensating" sequence of points in the muscle phase space. If the vestibular coordinates are directly used for specifying where the muscles should go, then the eyeball will end up in the wrong place, much as Roger's arm ends up in the wrong place if visual coordinates are not transformed into the coordinates of his motor space. For simplicity, the diagram assumes that the head is moving purely horizontally, so that the input maximally registers yaw, as opposed to pitch and roll.
The vestibular semicircular canals (A, anterior; P, posterior; H, horizontal) can be characterized by their rotational axes. For the motor system, the rotational axes of the eye correspond to the pull of extraocular muscles (LR, lateral rectus; MR, medial rectus; SR, superior rectus; IR, inferior rectus; SO, superior oblique; 10, inferior oblique). The rotational axes of the vestibular semicircular canals and the extraocular muscles constitute two built-in frames of reference for the CNS to measure a head movement and to execute a compensatory gaze-shift. (From Pellionisz (1985). In Adaptive Mechanisms in Gaze Control, ed. A. Berthoz and G. Melvill Jones. Copyright © 1985 by Elsevier Science Publishing Co., Inc.)
There are six extraocular muscles, so the muscle phase space is six-dimensional, and the muscle vector will have as its components the points on the axes for each muscle. Where any muscle is on its axis will be a function of its degree of contraction from a standard position, say when the eyeball is positioned directly ahead. Experimental data are available expressing the relation between muscle contraction and eyeball rotation (Volkmann 1869) and describing the excitatory sensitivity-axes for each vestibular canal (Blanks et al. 1975). Given these data, it is therefore possible to describe the vestibular phase space and the oculomotor phase space for a given head rotation. This is the basis for the quantitative description in figure 10.10.
The sensory and motor systems of coordinates of the VOR, intrinsic to CNS function, as defined by the vesribular matrix and eye muscle matrix. The directions in threedimensional XYZ space of the unit rotational axes, belonging to individual eye muscle contractions, are shown on the left; that is, this illustration shows how the eyeball moves in 3-space relative to the activity of a given muscle. The excitatory activation-axes of the combined semicircular canals of the two vestibuli are shown on the right. These two frames of reference therefore delimit the phase spaces within which the nervous system must function such that gaze control is achieved. To facilitate visual perception of the three-dimensional directions of the axes, their orthogonal projection to the XYZ plane is also indicated. The numerical values are based on anatomical data. (from Pellionisz (1985). In Adaptive Mechanisms in Gaze Control, ed. A. Berthoz and G. Melvill Jones. Copyright © 1985 by Elsevier Science Publishing Co., Inc.)
_________
According to the tensor network theory, there ought to be a tensorial transformation of the vestibular vector into the oculomotor vector. The Pellionisz-Llinas hypothesis is that a tensorial transformation takes place at each of the three synaptic levels, the last of which transforms a premotor vector into a motor vector that tells the muscles what the position in muscle phase space should be-in other words, how much each muscle should contract (figure 10.11). Since we can figure out what the positions of the eyeball should be given the position of the head, we can determine the tensorial transformations needed. Then we can work backward and figure out what the participating neurons at each stage should be doing. This in turn can be tested by seeing whether the neurons really do behave as the hypothesis says they should. As more is discovered about the neuronal basis, the basic hypothesis may be corrected and elaborated, and thus
