TENSOR NETWORK THEORY AND ITS APPLICATION FOR COMPUTER MODELING OF THE METAORGANIZATION OF SENSORIMOTOR HIERARCHIES OF GAZE
A. J. Pellionisz
Dept. Physiol. & Biophys. New York Univ. Med. Ctr. NY. 10016
THE CHALLENGE Neuronal networks are, in fact, used for "computations" in living organisms, producing what we call brain function (eg. sensorimotor coordination and intelligent representation). Based on what principle does the Central Nervous System (CNS) accomplish these tasks, and whether its mathematical understanding and subsequent or simultaneous technological imple- mentation will lead to utilizable socioeconomical applications, are questions increasingly in the forefront of the interest of neuroscience community at large1-3, and of its special field of brain theory, which is intimately tied to the artificial intelligence community and computer science and industry4-7. Activities range from mathematical analysis8-11 to rehabilitation medicine12,14. The overlap of neuroscience with other disciplines created interdisciplinary subfields; Neurobotics3,13,14, Neurophysics16-18, & Neurophilosophy19. The new scientific revolution attracts neuroscientists spanning from molecular biologists20,21 through mathematicians, engineers and physicists to philosophers. The implications warrant an increasing awareness of their vital importance by various government-agencies worldwide. A GEOMETRICAL APPROACH TO BRAIN FUNCTION: TENSOR NETWORK THEORY Motivated by the need of functionally interpreting the structure of existing neuronal networks22, such as those in the cerebellum, this author strives for finding the basic general principle of the organization of "neuronal networks", and gaining a conceptual and formal grasp on what they "compute". The approach exposed here concentrates on sensorimotor neuronal networks, (as in the cerebellum), and on the mathematical question of the axioms of their computations. Tensor network theory of the central nervous system may be summarized 1,2,14,15 by stating its axiom that the brain relates to the external world by expressing physical objects (invariants), both in a sensory and motor manner, in systems of coordinates that are intrinsic to the organism. Such general, typically non-orthogonal and overcomplete, frames of reference are physically obvious in sensory and motor parts of the CNS. Sensory and motor representation is identified in tensor network theory by covariant vectors 23 (with measurement-type orthogonal-projection components) and contravariant vectors (with physically executable parallelogram-type components), respectively. Thus, the metric tensor operation, which transforms these representations to one another was identified as a basic functional characteristics of sensorimotor networks, as elaborated for the cerebellum29. Beyond offering a formalism for describing neuronal computations of intrinsic vector- components of physical invariants, this approach conceptually features brain function as comprising functional geometries (via metric tensors, implemented by neuronal networks) in the internal CNS representation-spaces, both in sensorimotor and connected manifolds. COMPUTER MODEL OF THE METAORGANIZATION OF GAZE SENSORIMOTOR HIERARCHIES A quantitative example of this approach is a tensor model of gaze. To maintain a stable image in fixation, head & eye must compensate for passive movements or, in tracking, for the movements of the target; An invariant is expressed in various intrinsic coordinates (Fig.1).
Table I.
Table I. Data, from computerized anatomy, to define coordinate systems intrinsic to tensorial expressions of gaze . Rotational axes of (A): a mammalian retinal sensory frame24, (B):frame of vestibular canals25, (C): a motor frame of eye muscles26, (D):neck-muscles27 (15/side). _____ How various vectorial expressions within and among these frames are transformed by the CNS is the subject of tensor network theory: A 3-step tensorial scheme was elaborated to transfer covariant sensory coordinates to contravariants in a different motor frame2,28,32: 1) Sensory metric tensor (gpr), transforming a covariant reception vector (sr) to contravariant perception (sp, lower and upper indexes denote co- and contravariants): sp=gpr.sf where gpr =|gpr|-1=|cos(½pr)|-1 whereÊ|cos(½pr)| is the table of cosines of angles among sensory unit-vectors. 2) Sensorimotor covariant embedding tensor (cip) transforming the sensory vector (sp) into covariant motor intention vector(mi). Covariant embedding is unique regardless a dimensional inconsistency of the sensory and motor space (including over- completeness2), but results in a non-executable expression23: mi = cip . sp where cip= ui. wp where ui and wp are the i-th sensory unit-vector and p-th motor unit-vector. 3) Motor metric tensor1,2,23 that converts intention mi to executable contravariants; me = gei . mi (where gei is computed as gpr was for sensory axes in 1). In case of overcompleteness, of either or both sensory and motor coordinate systems (as in A,C,D in Fig. 1), tensor network theory hypothesizes23 that the CNS uses the Moore- Penrose generalized inverse (MP) of the unique covariant metric tensor2,15,30: gj,k= Summam {1/Lm+ . |Em > < Em|}, where Em and Lm+ are the m-th Eigenvector of gj,k and its Eigenvalue (replaced by 1 if it was 0). This 3-step scheme is used to compute tensors of a sensorimotor reflex2,28. For the 4 gaze reflexes of Fig.1, each expressing an invariant both in a sensory and a motor frame, the above calculation yields tensor-matrices as shown (by patch-diagrams only) in Fig.2.
Figure 1.
Fig. 1. Coordinates intrinsic to gaze sensorimotor neuronal networks. Gaze is expressed by rotations of the head & eye via neck & eye muscles, so that they compensate for rotations measured by the retinal ganglion cells and by the vestibular semicircular canals. Both the twin (retinal and vestibular) sensory apparatus and the twin (oculomotor and neck-motor) executor systems operate along rotational axes determined by the structure of the organism.To express gaze, neuronal networks must measure and produce physical invariants (movements) in these typically non-orthogonal, overcomplete intrinsic frames of reference, by covariant sensory and contravariant motor vectors. Since the frames have been established by quantitative anatomy (cf. Table l.), the task we face is to quantitatively interpret how the CNS establishes relationships among various vectorial representations of a physical invariant such as gaze. Tensor transformations yield a general interpretation & calculation, which tensor-matrices are implemented in the CNS by the system of interconnections in neuronal networks .
Figure 2.
Fig.2.Metaorganization, in 6 developmental steps, of sensorimotor reflexes involved in gaze. 3 neuronal networks, required for tensor-transformations in each sensorimotor reflex, eg. from vestibular- to neck-motor vector in (1), were calculated by the 3-step tensor scheme 2,28,31. Resulting tensor-matrices are shown by 3 patch-diagrams in each sensorimotor reflex-arc; VCR(1), RCR(2), ROR(3), VOR(4). These networks are to develop in a definite sequence; in the VCR(1) the motor metric, sensorimotor embedding and sensory metric develop, as described by metaorganization31. RCR(2) builds hierarchically on the existing neck-motor metric, and the retinal metric is used also for ROR(3). VOR(4) is built on top of this hierarchy, using the vestibular metric available from VCR. Since VOR is the only gaze reflex which is not a closed-loop sensorimotor system28, its development must use the already available RCR,VCR,ROR networks. (The oculo-ocular & cervico-collic motor metrics whose development precede those of the 4 gaze reflexes are also indicated in the last scheme).
Figure 3. Fig. 3. Tensor network model of gaze, shown by a schematic visualization of the computed 8 network-matrices, necessary for tensor-transformations in the multi- sensory-multimotor apparatus. The 30X30=900 elements of the motor metric tensor (a Moore-Penrose generalized inverse) is to be implemented by the cerebellar neuronal network. Metaorganization 31, by offering a theory for the development and construction of coordinated overcomplete, multisensory and multimotor apparatus, appears to yield both mathematical and neurobiological advantages. _____ As for theory and implementation, advantages result from its employing the MP formula, which a) yields the proper inverse if the space is complete, b) yields a least-squares minimum-energy formula, c) can be generated by the CNS via the process of metaorganization (also yielding the sensory metric and sensorimotor covariant embedding networks), by the utilization of reverberative oscillations31, d) the theoretical prediction has been experimentally shown to conform with the CNS in gaze control32 and coordination of human arm33. As for Neuroscience, tensor network theory may be useful by functionally interpreting existing neuronal networks. As suggested, the proposed metric-type function can be imple- mented for sensory modalities by the tectum30, for motor vectors by the cerebellum29. Structural features of the proposed tensor-transformation matrices in sensorimotor reflexes, eg. the three tensor-transformations in the VCR and VOR, appear to match structural properties of the CNS, where eg. vestibular signals are known to be transformed from the semicircular canals to vestibular nuclei, from there to premotor nuclei, and to oculomotor nuclei, before reaching eye muscles28,32. While it may require an enormous cooperative effort, quantitative anatomy, experimental network analysis and tensor network theory may gradually reveal not only quantitative operational features of neuronal circuitries, but also basic principles of brain function. This work starts on the proving ground of Brain Theory, sensorimotor coordination, where the physical entities that are the objectives of neuronal computation are most evident. Principles and techniques learnt from these studies, if truly general, could be helpful towards understanding intelligent representations in the neocortex. FUNCTIONAL GEOMETRIES IN CONNECTED CNS REPRESENTATION HYPERSPACES Metaorganization of gaze networks is an example for creating sensory- and motor metric- type networks. They comprise functional geometries to match the physical geometry of the structure of sensory and motor apparatus. Neuronal networks in the brain , however, incor- porate functional geometries that not only passively react to given physical geometries by compensating for modifications occurring in their relation to the environment, as in gaze, but impose intelligent function on both the sensorimotor apparatus, and the world15. Such active modification, to be intelligent, requires the brain to comprise a functional geometry, a world model, with a homeomorphic geometry. Intelligent functional geometries of the CNS are presently largely unexplored, but are expected to transgress the boundaries of Euclidean or Riemannian manifolds15. Thus, a study of sensorimotor spaces that are directly connected to known external structural geometries may be essential homework. Then the generalized prin- ciple of metaorganization and the general formalism of tensor network theory may be applied to tackle how intelligent geometries develop in the CNS, or can be developed to extend them. REFERENCES 1 Pellionisz, A.J. In:Brain Theory (Palm G, Aertsen A,eds) Springer, 114-135(1986) 2 Pellionisz, A.J. J. Theoretical Biology 110:353-375 (1984) 3 Loeb, J.W. Trends in Neuroscience 5:203-204 (1983) 4 Kohonen, T. Associative Memory, Springer Verlag Heidelb.-New York-Berlin (1977) 5 Anderson, J.R., G.H.Bower Human Associative Memory, Winston, Wash. DC (1973) 6 Fukushima, K. Biological Cybernetics 36:193-202 (1980) 7 Palm, G., A. Aertsen, Brain Theory , Springer, Heidelberg, (1986) 8 Marr, D., T. Poggio , Science 194:283-287 (1976) 9 Amari, S.Biological Cybernetics 26:175-185 (1977) 10 Palm, G. Neural Assemblies, Springer, Heidelberg-New York-Berlin (1982) 11 Grossberg, S. Studies of Mind & Brain, D.Reidel, Dordrecht (1982) 12 Mann, R.W. Ann Biomedical Engineering 9:1-43 (1981) 13 Hogan, N. J. Neuroscience 4(11): 2745-2754 (1984) 14 Pellionisz, A.J.IEEE Conf. on Systems Man & Cybernetics. pp.411-414 (1985) 15 Pellionisz, A.J. J. Theoretical Neurobiology 2(3):185-211 (1983) 16 Cooper, L.N.In:Coll. Prop. of Physic. Syst. Acad. Press New York, pp.252-264(1974) 17 Hopfield, J.J., D.I. Feinstein, R.G. Palmer, Nature 304:158-159 (1983) 18 Scott, A.C. Neurophysics, Wiley, Interscience (1977) 19 Churchland, P.S. Neurophilosophy, MIT Press (1986) 20 Crick, F. Scientific American 241:219-232 (1979) 21 Edelman, G.M., V.B. Mountcastle, The Mindful Brain, MIT Press (1978) 22 Pellionisz, A.J., J. Szentagothai, Brain Research 49:83-99 (1973) 23 Pellionisz, A.J., R. Llinas, Neuroscience 5:1125-1136 (1980) 24 Oyster, C.W. J. Physiology (Lond.) 199:613-635 (1968) 25 Blanks,R.H.I., I.S.Curthoys, C.H.Markham, Acta Otolaryng. 80:185-196(1975) 26 Ezure,K., W. Graf, Neuroscience 12:95-109 (1984) 27 Baker,J., J. Goldberg, B.W. Peterson, J. Neurophysiology 54:735-756(1985) 28 Pellionisz, A.J. In:Adapt.Mech.of Gaze Contr. Berthoz A(ed)Elsevier 281-296(1985) 29 Pellionisz, A.J. In: Cerebellar Funct. Bloedel JR et al.(eds)Springer201-229(1985) 30 Pellionisz, A.J. In:Visuomotor Coordination ,(Lara, R, Arbib M. Eds) pp.1-20(1983) 31 Pellionisz, A.J., R. Llinas, Neuroscience 16:245-274 (1985) 32 Pellionisz, A.J., B.W. Peterson In: Control of Head Movements, Acad. Press (1986) 33 Gielen, C.C.A.M., E.J. van Zuylen, Neuroscience 17:527-539 (1986) Supported by Grant NS 22999 from NINCDS
Table I.
Figure 1.
Figure 2.
Figure 3.