CiteULike: Tag neural

Predicting Human Interactive Learning by Regret-Driven Neural Networks

Davide Marchiori — 2008-02-21T21:44:00-00:00

Science, Vol. 319, No. 5866. (22 February 2008), pp. 1111-1113.

Much of human learning in a social context has an interactive nature: What an individual learns is affected by what other individuals are learning at the same time. Games represent a widely accepted paradigm for representing interactive decision-making. We explored the potential value of neural networks for modeling and predicting human interactive learning in repeated games. We found that even very simple learning networks, driven by regret-based feedback, accurately predict observed human behavior in different experiments on 21 games with unique equilibria in mixed strategies. Introducing regret in the feedback dramatically improved the performance of the neural network. We show that regret-based models provide better predictions of learning than established economic models. 10.1126/science.1151185

What are artificial neural networks?

Anders Krogh — 2008-02-08T00:40:34-00:00

Nature Biotechnology, Vol. 26, No. 2., pp. 195-197.

Reducing the Dimensionality of Data with Neural Networks

GE Hinton — 2006-07-28T15:16:42-00:00

Science, Vol. 313, No. 5786. (28 July 2006), pp. 504-507.

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such "autoencoder" networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. 10.1126/science.1127647

COMPUTER SCIENCE: New Life for Neural Networks

Garrison Cottrell — 2006-07-29T02:23:16-00:00

Science, Vol. 313, No. 5786. (28 July 2006), pp. 454-455.

10.1126/science.1129813

Mixture segmentation and background suppression in chemosensor arrays with a model of olfactory bulb-cortex interaction

B Raman — 2008-05-12T14:14:28-00:00

Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005. IJCNN '05, Vol. 1 (2005), pp. 131-136.

We present a model of olfactory bulb-cortex interaction for the purpose of mixture processing with gas sensor arrays. The olfactory bulb is modeled with a neurodynamic model whose lateral inhibitory connections are learned through a modified Hebbian-anti-Hebbian rule. Bulbar outputs are then projected in a non-topographic fashion onto the olfactory cortex. Associational connections within cortex using Hebbian learning form a content addressable memory. Finally, inhibitory feedback from cortex is used to modulate bulbar activity. Depending on the form of feedback, Hebbian or anti-Hebbian, the model is able to perform background suppression or mixture segmentation. The model is validated on experimental data from a gas sensor array.

Obstacle avoidance of a mobile robot using hybrid learning approach

Meng Er — 2005-12-17T18:47:22-00:00

Industrial Electronics, IEEE Transactions on, Vol. 52, No. 3. (2005), pp. 898-905.

in this paper, a hybrid learning approach for obstacle avoidance of a mobile robot is presented. the key features of the approach are, firstly, innate hardwired behaviors which are used to bootstrap learning in the mobile robot system. a neuro-fuzzy controller is developed from a pre-wired or innate controller based on supervised learning in a simulation environment. the fuzzy inference system has been constructed based on the generalized dynamic fuzzy neural networks learning algorithm of Wu and Er, whereby structure and parameters identification are carried out automatically and simultaneously. Secondly, the neuro-fuzzy controller is capable of re-adapting in a new environment. After carrying out the learning phase on a simulated robot, the controller is implemented on a real robot. A reinforcement learning method based on the fuzzy actor-critic learning algorithm is employed so that the system can re-adapt to a new environment without human intervention. In this phase, the structure of the fuzzy inference system and the parameters of the antecedent parts of fuzzy rules are frozen, and reinforcement learning is applied to further tune the parameters in the consequent parts of the fuzzy rules. Through the hybrid learning approach, an efficient and compact neuro-fuzzy system is generated for obstacle avoidance of a mobile robot in the real world.

Intelligent optimal control of single-link flexible robot arm

Rong-Jong Wai — 2005-12-17T18:46:55-00:00

Industrial Electronics, IEEE Transactions on, Vol. 51, No. 1. (2004), pp. 201-220.

This paper addresses the design and properties of an intelligent optimal control for a nonlinear flexible robot arm that is driven by a permanent-magnet synchronous servo motor. First, the dynamic model of a flexible robot arm system with a tip mass is introduced. When the tip mass of the flexible robot arm is a rigid body, not only bending vibration but also torsional vibration are occurred. In this paper, the vibration states of the nonlinear system are assumed to he unmeasurable, i.e., only the actuator position can be acquired to feed into a suitable control system for stabilizing the vibration states indirectly. Then, an intelligent optimal control system is proposed to control the motor-mechanism coupling system for periodic motion. In the intelligent optimal control system a fuzzy neural network controller is used to learn a nonlinear function in the optimal control law, and a robust controller is designed to compensate the approximation error. Moreover, a simple adaptive algorithm is proposed to adjust the uncertain bound in the robust controller avoiding the chattering phenomena. The control laws of the intelligent optimal control system are derived in the sense of optimal control technique and Lyapunov stability analysis, so that system-tracking stability can be guaranteed in the closed-loop system. In addition, numerical simulation and experimental results are given to verify the effectiveness of the proposed control scheme.

New approach to intelligent control systems with self-exploring process

Liang-Hsuan Chen — 2005-12-17T18:46:34-00:00

Systems, Man and Cybernetics, Part B, IEEE Transactions on, Vol. 33, No. 1. (2003), pp. 56-66.

This paper proposes an intelligent control system called self-exploring-based intelligent control system (SEICS). The SEICS is comprised of three basic mechanisms, namely, controller, performance evaluator (PE), and adaptor. The controller is constructed by a fuzzy neural network (FNN) to carry out the control tasks. The PE is used to determine whether or not the controller's performance is satisfactory. The adaptor, comprised of two elements, action explorer (AE) and rule generator (RG), plays the main role in the system for generating new control behaviors in order to enhance the control performance. AE operates through a three-stage self-exploration process to explore new actions, which is realized by the multiobjective genetic algorithm (GA). The RG transforms control actions to fuzzy rules based on a numerical method. The application of the adaptor can make a control system more adaptive in various environments. A simulation of robotic path-planning is used to demonstrate the proposed model. The results show that the robot reaches the target point from the start point successfully in the lack-of-information and changeable environments.

Robust adaptive control of robot manipulators using generalized fuzzy neural networks

Meng Er — 2005-12-17T18:46:13-00:00

Industrial Electronics, IEEE Transactions on, Vol. 50, No. 3. (2003), pp. 620-628.

This paper presents a robust adaptive fuzzy neural controller (AFNC) suitable for motion control of multilink robot manipulators. The proposed controller has the following salient features: (1) self-organizing fuzzy neural structure, i.e., fuzzy control rules can be generated or deleted automatically according to their significance to the control system and the complexity of the mapped system and no predefined fuzzy rules are required; (2) fast online learning ability, i.e., no prescribed training models are needed for online learning and weights of the fuzzy neural controller are modified without any iterations; (3) fast convergence of tracking errors, i.e., manipulator joints can track the desired trajectories very quickly; (4) adaptive control, i.e., structure and parameters of the AFNC can be self-adaptive in the presence of disturbances to maintain high control performance; and (5) robust control, where asymptotic stability of the control system is established using the Lyapunov theorem. Experimental evaluation conducted on an industrial selectively compliant assembly robot arm demonstrates that excellent tracking performance can be achieved under time-varying conditions.

Adaptive control of robot manipulators using fuzzy neural networks

Yang Gao — 2005-12-17T18:45:51-00:00

Industrial Electronics, IEEE Transactions on, Vol. 48, No. 6. (2001), pp. 1274-1278.

This paper presents an adaptive fuzzy neural controller suitable for multilink manipulators motion control. The proposed controller has the following salient features: (1) self-organizing fuzzy neural structure; (2) online learning of the robot dynamics; (3) fast convergence of tracking error; and (4) adaptive control. Computer simulation results of a two-link manipulator demonstrate that excellent tracking performance can be achieved under external disturbances

A fast approach for automatic generation of fuzzy rules by generalized dynamic fuzzy neural networks

Shiqian Wu — 2005-12-17T18:45:28-00:00

Fuzzy Systems, IEEE Transactions on, Vol. 9, No. 4. (2001), pp. 578-594.

A fast approach for automatically generating fuzzy rules from sample patterns using generalized dynamic fuzzy neural networks (GD-FNNs) is presented. The GD-FNN is built based on ellipsoidal basis functions and functionally is equivalent to a Takagi-Sugeno-Kang fuzzy system. The salient characteristics of the GD-FNN are: (1) structure identification and parameters estimation are performed automatically and simultaneously without partitioning input space and selecting initial parameters a priori; (2) fuzzy rules can be recruited or deleted dynamically; (3) fuzzy rules can be generated quickly without resorting to the backpropagation (BP) iteration learning, a common approach adopted by many existing methods. The GD-FNN is employed in a wide range of applications ranging from static function approximation and nonlinear system identification to time-varying drug delivery system and multilink robot control. Simulation results demonstrate that a compact and high-performance fuzzy rule-base can be constructed. Comprehensive comparisons with other latest approaches show that the proposed approach is superior in terms of learning efficiency and performance

Position/force control of robot manipulators for geometrically unknown objects using fuzzy neural networks

K Kiguchi — 2005-12-17T18:45:06-00:00

Industrial Electronics, IEEE Transactions on, Vol. 47, No. 3. (2000), pp. 641-649.

In order to carry out the tasks of grinding, deburring, polishing or wiping, the end-effector of the robot manipulator has to follow the contour of an object. In this paper, the authors propose a fuzzy vector method, which enables the controller to deal efficiently with force sensor signals which include noise and/or unknown vibrations caused by the working tool, to search the direction of the constraint surface of an unknown object

Fuzzy neural network approaches for robotic gait synthesis

Jih-Gau Juang — 2005-12-17T18:44:44-00:00

Systems, Man and Cybernetics, Part B, IEEE Transactions on, Vol. 30, No. 4. (2000), pp. 594-601.

In this paper, a learning scheme using a fuzzy controller to generate walking gaits is developed. The learning scheme uses a fuzzy controller combined with a linearized inverse biped model. The controller provides the control signals at each control time instant. The algorithm used to train the controller is “backpropagation through time”. The linearized inverse biped model provides the error signals for backpropagation through the controller at control time instants. Given prespecified constraints such as the step length, crossing clearance, and walking speed, the control scheme can generate the gait that satisfies these constraints. Simulation results are reported for a five-link biped robot

Intelligent tracking control for robot manipulator including actuator dynamics via TSK-type fuzzy neural network

Rong-Jong Wai — 2005-12-17T18:44:24-00:00

Fuzzy Systems, IEEE Transactions on, Vol. 12, No. 4. (2004), pp. 552-560.

In this paper, a Takagi-Sugeno-Kang-type fuzzy-neural-network control (T-FNNC) scheme is constructed for an n-link robot manipulator to achieve high-precision position tracking. According to the concepts of mechanical geometry and motion dynamics, the dynamic model of an n-link robot manipulator including actuator dynamics is introduced initially. However, it is difficult to design a suitable model-based control scheme due to the uncertainties in practical applications, such as friction forces, external disturbances and parameter variations. In order to cope with this problem, a T-FNNC system without the requirement of prior system information and auxiliary control design is investigated to the joint position control of an n-link robot manipulator for periodic motion. In this model-free control scheme, a five-layer fuzzy-neural-network is utilized for the major control role, and the adaptive tuning laws of network parameters are established in the sense of projection algorithm and Lyapunov stability theorem to ensure the network convergence as well as stable control performance. In addition, experimental results of a two-link robot manipulator actuated by dc servomotors are provided to verify the effectiveness and robustness of the proposed T-FNNC methodology.

Fuzzy neural network quadratic stabilization output feedback control for biped robots via H/sub /spl infin// approach

Zhi Liu — 2005-12-17T18:44:08-00:00

Systems, Man and Cybernetics, Part B, IEEE Transactions on, Vol. 33, No. 1. (2003), pp. 67-84.

A novel fuzzy neural network (FNN) quadratic stabilization output feedback control scheme is proposed for the trajectory tracking problems of biped robots with an FNN nonlinear observer. First, a robust quadratic stabilization FNN nonlinear observer is presented to estimate the joint velocities of a biped robot, in which an H/sub /spl infin// approach and variable structure control (VSC) are embedded to attenuate the effect of external disturbances and parametric uncertainties. After the construction of the FNN nonlinear observer, a quadratic stabilization FNN controller is developed with a robust hybrid control scheme. As the employment of a quadratic stability approach, not only does it afford the possibility of trading off the design between FNN, H/sub /spl infin// optimal control, and VSC, but conservative estimation of the FNN reconstruction error bound is also avoided by considering the system matrix uncertainty separately. It is shown that all signals in the closed-loop control system are bounded.

Pedestrian-Behavior-Based Mobile Agent Control in Intelligent Space

PT Szemes — 2005-12-17T18:43:35-00:00

Instrumentation and Measurement, IEEE Transactions on, Vol. 54, No. 6. (2005), pp. 2250-2257.

This paper investigates a human walking behavior-based mobile robot control system. The Intelligent Space (iSpace) is a distributed sensory system, which is the background infrastructure to observe human walking in a limited area. The observation of human walking behavior is applied to train fuzzy-neural networks (FNN). The trained FNNs are applied to approximate the obstacle avoidance behavior of human walking. The paper introduces the iSpace and the mobile agents, which are mobile robots, utilizing the intelligence of the iSpace. The observed and trained human walking behaviors are applied to control the mobile agent in a human-robot shared environment. Experimental results demonstrate the effectiveness of the FNN-based control system.

Robot adaptivity

Carme Torras — 2005-12-17T18:35:07-00:00

Robotics and Autonomous Systems, Vol. 15, No. 1-2. (July 1995), pp. 11-23.

Adaptivity is a crucial capability for survival in the biological world. Analogously, in order for robots to carry out their tasks autonomously in unstructured environments, it is essential that they be well-adapted to their surroundings. The most common way to endow robots with this capability is through the use of neural learning techniques. The main such techniques are reviewed in this paper, together with their use in both mobile robots and manipulator arms. Not only the advantages, but also the limitations of neural adaptivity are pointed out.

Vision-based robot positioning using neural networks

Gordon Wells — 2005-12-17T18:34:07-00:00

Image and Vision Computing, Vol. 14, No. 10. (December 1996), pp. 715-732.

Most vision-based robot positioning techniques rely on analytical formulations of the relationship between the robot pose and the projected image coordinates of several geometric features of the observed scene. This usually requires that several simple features such as points, lines or circles be visible in the image, which must either be unoccluded in multiple views or else part of a 3D model. Featurematching algorithms, camera calibration, models of the camera geometry and object feature relationships are also necessary for pose determination. These steps are often computationally intensive and error-prone, and the complexity of the resulting formulations often limits the number of controllable degrees of freedom. We provide a comparative survey of existing visual robot positioning methods, and present a new technique based on neural learning and global image descriptors which overcomes many of these limitations. A feedforward neural network is used to learn the complex implicit relationship between the pose displacements of a 6-dof robot and the observed variations in global descriptors of the image, such as geometric moments and Fourier descriptors. The trained network may then be used to move the robot from arbitrary initial positions to a desired pose with respect to the observed scene. The method is shown to be capable of positioning an industrial robot with respect to a variety of complex objects with an acceptable precision for an industrial inspection application, and could be useful in other real-world tasks such as grasping, assembly and navigation.

BISMARC: a biologically inspired system for map-based autonomous rover control

Terry Huntsberger — 2005-12-17T18:33:24-00:00

Neural Networks, Vol. 11, No. 7-8. (11 October 1998), pp. 1497-1510.

As the complexity of the missions to planetary surfaces increases, so too does the need for autonomous rover systems. This need is complicated by the power, mass and computer storage restrictions on such systems (Miller, D. P. (1992). Reducing software mass through behaviour control. In Proceedings SPIE conference on cooperative intelligent robotics in space III (Vol. 1829, pp. 472-475, 1992). Boston, MA. To address these problems, we have recently developed a system called BISMARC (Biologically Inspired System for Map-based Autonomous Rover Control) for planetary missions involving multiple small, lightweight surface rovers (Huntsberger, T. L. (1997). Autonomous multirover system for complex planetary retrieval operations. In P. S. Schenker, and G. T. McKee (Eds.), Proceedings SPIE symposium on sensor fusion and decentralized control in autonomous robotic systems (pp. 221-227). Pittsburgh, PA). BISMARC is capable of cooperative planetary surface retrieval operations such as a multiple cache recovery mission to Mars. The system employs autonomous navigation techniques, behavior-based control for surface retrieval operations, and an action selection mechanism based on a modified form of free flow hierarchy (Rosenblatt, J. K. and Payton, D. W. (1989). A fine-grained alternative to the subsumption architecture for mobile robot control. In Proceedings IEEE/INNS joint conference on neural networks (pp. 317-324). Washington, DC). This paper primarily describes the navigation and map-mapping subsystems of BISMARC. They are inspired by some recent studies of London taxi drivers indicating that the right hippocampal region of the brain is activated for path planning but not for landmark identification (Maguire, E. A. et al. (1997). Recalling routes around London: activation of the right hippocampus in taxi drivers. Journal of Neuroscience, 17(18), 7103-7110). We also report the results of some experimental studies of simulated navigation in planetary environments.

Topography of cognition: parallel distributed networks in primate association cortex.

PS Goldman-Rakic — 2006-03-02T00:47:18-00:00

Annu Rev Neurosci, Vol. 11 (1988), pp. 137-156.

Structure and Dynamics of Recurrent Neuromodules

Frank Pasemann — 2007-09-17T21:40:10-00:00

Theory in Biosciences, Vol. 117 (1998), pp. 1-17.

The article calls attention to complex dynamical phenomena in artificial neural systems, which are - as is asserted - of relevance also for understanding biological brain functions. Examples of various dynamical effects (hysteresis, oscillations, deterministic chaos, synchronization and coherence) are discussed in terms of the discrete dynamics of small recurrent networks. The relevance of a dynamical systems approach for understanding the emergence of higher level information processing or...

Dynamical Motifs: Building Blocks of Complex Dynamics in Sparsely Connected Random Networks

Valentin Zhigulin — 2007-04-03T15:07:16-00:00

Physical Review Letters, Vol. 92, No. 23. (2004)

Spatiotemporal network dynamics is an emergent property of many complex systems that remains poorly understood. We suggest a new approach to its study based on the analysis of dynamical motifs—small subnetworks with periodic and chaotic dynamics. We simulate randomly connected neural networks and, with increasing density of connections, observe the transition from quiescence to periodic and chaotic dynamics. This transition is explained by the appearance of dynamical motifs in the structure of these networks. We also observe domination of periodic dynamics in simulations of spatially distributed networks with local connectivity and explain it by the absence of chaotic and the presence of periodic motifs in their structure.

Plasticity in single neuron and circuit computations

Alain Destexhe — 2005-10-18T09:14:16-00:00

Nature, Vol. 431, No. 7010. (14 October 2004), pp. 789-795.

Well-being and affective style: neural substrates and biobehavioural correlates.

RJ Davidson — 2008-05-22T12:03:44-00:00

Philosophical transactions of the Royal Society of London. Series B, Biological sciences, Vol. 359, No. 1449. (29 September 2004), pp. 1395-1411.

One of the most salient features of emotion is the pronounced variability among individuals in their reactions to emotional incentives and in their dispositional mood. Collectively, these individual differences have been described as affective style. Recent research has begun to dissect the constituents of affective style. The search for these components is guided by the neural systems that instantiate emotion and emotion regulation. In this article, this body of research and theory is applied specifically to positive affect and well-being. The central substrates and peripheral biological correlates of well-being are described. A resilient affective style is associated with high levels of left prefrontal activation, effective modulation of activation in the amygdala and fast recovery in response to negative and stressful events. In peripheral biology, these central patterns are associated with lower levels of basal cortisol and with higher levels of antibody titres to influenza vaccine. The article concludes with a consideration of whether these patterns of central and peripheral biology can be modified by training and shifted toward a more salubrious direction.

Functional Topology Classification of Biological Computing Networks

Pablo Blinder — 2005-12-14T12:03:20-00:00

Natural Computing, Vol. 4, No. 4. (September 2005), pp. 339-361.

Comment on "Network motifs: simple building blocks of complex networks" and "Superfamilies of evolved and designed networks".

Y Artzy-Randrup — 2004-11-22T00:17:30-00:00

Science, Vol. 305, No. 5687. (20 August 2004)

Structure–function relationship in complex brain networks expressed by hierarchical synchronization

Changsong Zhou — 2007-06-29T02:46:40-00:00

New J. Phys., Vol. 9, No. 6. (June 2007), 178.

The small world of modular networks

Raj Pan — 2008-03-06T07:14:23-00:00

(25 Feb 2008)

A large number of networks occurring in reality exhibit modular structure. Such systems can be decomposed into distinct compartments whose members are highly inter-connected in comparison to the density of connections between compartments. Many of these networks also possess the small-world property, i.e., coexistence of high communication efficiency with strong local clustering among their elements. Although both these properties confer certain advantages to the corresponding network, especially in an evolutionary context, until now they have been considered to be independent features of the system. In this paper, we show through a simple model that the small-world property arises directly as a result of the modular configuration of such networks. The proposed network model, composed of sparsely connected modules, differs from previous models for small-world networks which typically assume connections to occur mostly among neighboring nodes on an underlying regular lattice with a few long-range links. We also establish a distinct dynamical signature for such modular networks, namely, the existence of two characteristic time scales in processes such as synchronization and diffusion, a significant difference from earlier small-world network models. This dichotomy between fast intra-modular dynamics and slow inter-modular dynamics is directly related to the topological structure of the model through the spectral behavior of the network Laplacian. By verifying the existence of similar features in the example of empirically determined cortico-cortical networks, we propose that the modular network model may better represent certain systems reported to have small-world properties.

Generalized integrate-and-fire models of neuronal activity approximate spike trains of a detailed model to a high degree of accuracy.

R Jolivet — 2005-02-02T00:43:21-00:00

J Neurophysiol, Vol. 92, No. 2. (August 2004), pp. 959-976.

We demonstrate that single-variable integrate-and-fire models can quantitatively capture the dynamics of a physiologically detailed model for fast-spiking cortical neurons. Through a systematic set of approximations, we reduce the conductance-based model to 2 variants of integrate-and-fire models. In the first variant (nonlinear integrate-and-fire model), parameters depend on the instantaneous membrane potential, whereas in the second variant, they depend on the time elapsed since the last spike [Spike Response Model (SRM)]. The direct reduction links features of the simple models to biophysical features of the full conductance-based model. To quantitatively test the predictive power of the SRM and of the nonlinear integrate-and-fire model, we compare spike trains in the simple models to those in the full conductance-based model when the models are subjected to identical randomly fluctuating input. For random current input, the simple models reproduce 70-80 percent of the spikes in the full model (with temporal precision of +/-2 ms) over a wide range of firing frequencies. For random conductance injection, up to 73 percent of spikes are coincident. We also present a technique for numerically optimizing parameters in the SRM and the nonlinear integrate-and-fire model based on spike trains in the full conductance-based model. This technique can be used to tune simple models to reproduce spike trains of real neurons.

Neural substrates of processing syntax and semantics in music

Stefan Koelsch — 2005-07-15T22:31:13-00:00

Current Opinion in Neurobiology, Vol. 15, No. 2. (April 2005), pp. 207-212.

Growing evidence indicates that syntax and semantics are basic aspects of music. After the onset of a chord, initial music-syntactic processing can be observed at about 150-400 ms and processing of musical semantics at about 300-500 ms. Processing of musical syntax activates inferior frontolateral cortex, ventrolateral premotor cortex and presumably the anterior part of the superior temporal gyrus. These brain structures have been implicated in sequencing of complex auditory information, identification of structural relationships, and serial prediction. Processing of musical semantics appears to activate posterior temporal regions. The processes and brain structures involved in the perception of syntax and semantics in music have considerable overlap with those involved in language perception, underlining intimate links between music and language in the human brain.

Swinging in the brain: shared neural substrates for behaviors related to sequencing and music

Petr Janata — 2008-02-08T01:02:50-00:00

Nat Neurosci, Vol. 6, No. 7. (July 2003), pp. 682-687.

Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits

Sen Song — 2006-11-30T10:24:54-00:00

PLoS Biology, Vol. 3, No. 3. (1 March 2005), e68.

How different is local cortical circuitry from a random network? To answer this question, we probed synaptic connections with several hundred simultaneous quadruple whole-cell recordings from layer 5 pyramidal neurons in the rat visual cortex. Analysis of this dataset revealed several nonrandom features in synaptic connectivity. We confirmed previous reports that bidirectional connections are more common than expected in a random network. We found that several highly clustered three-neuron connectivity patterns are overrepresented, suggesting that connections tend to cluster together. We also analyzed synaptic connection strength as defined by the peak excitatory postsynaptic potential amplitude. We found that the distribution of synaptic connection strength differs significantly from the Poisson distribution and can be fitted by a lognormal distribution. Such a distribution has a heavier tail and implies that synaptic weight is concentrated among few synaptic connections. In addition, the strengths of synaptic connections sharing pre- or postsynaptic neurons are correlated, implying that strong connections are even more clustered than the weak ones. Therefore, the local cortical network structure can be viewed as a skeleton of stronger connections in a sea of weaker ones. Such a skeleton is likely to play an important role in network dynamics and should be investigated further.

Effects of topology on network evolution

Panos Oikonomou — 2007-05-07T23:18:02-00:00

Nat Phys, Vol. 2, No. 8. (2006), pp. 532-536.

The application of graph theoretical analysis to complex networks in the brain

Jaap Reijneveld — 2007-09-28T12:57:21-00:00

Clinical Neurophysiology, Vol. In Press, Corrected Proof

Considering the brain as a complex network of interacting dynamical systems offers new insights into higher level brain processes such as memory, planning, and abstract reasoning as well as various types of brain pathophysiology. This viewpoint provides the opportunity to apply new insights in network sciences, such as the discovery of small world and scale free networks, to data on anatomical and functional connectivity in the brain. In this review we start with some background knowledge on the history and recent advances in network theories in general. We emphasize the correlation between the structural properties of networks and the dynamics of these networks. We subsequently demonstrate through evidence from computational studies, in vivo experiments, and functional MRI, EEG and MEG studies in humans, that both the functional and anatomical connectivity of the healthy brain have many features of a small world network, but only to a limited extent of a scale free network. The small world structure of neural networks is hypothesized to reflect an optimal configuration associated with rapid synchronization and information transfer, minimal wiring costs, resilience to certain types of damage, as well as a balance between local processing and global integration. Eventually, we review the current knowledge on the effects of focal and diffuse brain disease on neural network characteristics, and demonstrate increasing evidence that both cognitive and psychiatric disturbances, as well as risk of epileptic seizures, are correlated with (changes in) functional network architectural features.

Organization, development and function of complex brain networks.

O Sporns — 2004-11-22T00:17:30-00:00

Trends Cogn Sci, Vol. 8, No. 9. (September 2004), pp. 418-425.

Recent research has revealed general principles in the structural and functional organization of complex networks which are shared by various natural, social and technological systems. This review examines these principles as applied to the organization, development and function of complex brain networks. Specifically, we examine the structural properties of large-scale anatomical and functional brain networks and discuss how they might arise in the course of network growth and rewiring. Moreover, we examine the relationship between the structural substrate of neuroanatomy and more dynamic functional and effective connectivity patterns that underlie human cognition. We suggest that network analysis offers new fundamental insights into global and integrative aspects of brain function, including the origin of flexible and coherent cognitive states within the neural architecture.

Structure and Dynamics of Random Recurrent Neural Networks

Hugues Berry — 2008-02-28T04:02:01-00:00

Adaptive Behavior, Vol. 14, No. 2. (1 June 2006), pp. 129-137.

Contrary to Hopfield-like networks, random recurrent neural networks (RRNN), where the couplings are random, exhibit complex dynamics (limit cycles, chaos). It is possible to store information in these networks through Hebbian learning. Eventually, learning "destroys" the dynamics and leads to a fixed point attractor. We investigate here the structural changes occurring in the network through learning. We show that a simple Hebbian learning rule organizes synaptic weight redistribution on the network from an initial homogeneous and random distribution to a heterogeneous one, where strong synaptic weights preferentially assemble in triangles. Hence learning organizes the network of the large synaptic weights as a "small-world" one 10.1177/105971230601400204

From sensation to cognition.

MM Mesulam — 2007-01-03T23:42:30-00:00

Brain, Vol. 121 ( Pt 6) (June 1998), pp. 1013-1052.

Sensory information undergoes extensive associative elaboration and attentional modulation as it becomes incorporated into the texture of cognition. This process occurs along a core synaptic hierarchy which includes the primary sensory, upstream unimodal, downstream unimodal, heteromodal, paralimbic and limbic zones of the cerebral cortex. Connections from one zone to another are reciprocal and allow higher synaptic levels to exert a feedback (top-down) influence upon earlier levels of processing. Each cortical area provides a nexus for the convergence of afferents and divergence of efferents. The resultant synaptic organization supports parallel as well as serial processing, and allows each sensory event to initiate multiple cognitive and behavioural outcomes. Upstream sectors of unimodal association areas encode basic features of sensation such as colour, motion, form and pitch. More complex contents of sensory experience such as objects, faces, word-forms, spatial locations and sound sequences become encoded within downstream sectors of unimodal areas by groups of coarsely tuned neurons. The highest synaptic levels of sensory-fugal processing are occupied by heteromodal, paralimbic and limbic cortices, collectively known as transmodal areas. The unique role of these areas is to bind multiple unimodal and other transmodal areas into distributed but integrated multimodal representations. Transmodal areas in the midtemporal cortex, Wernicke's area, the hippocampal-entorhinal complex and the posterior parietal cortex provide critical gateways for transforming perception into recognition, word-forms into meaning, scenes and events into experiences, and spatial locations into targets for exploration. All cognitive processes arise from analogous associative transformations of similar sets of sensory inputs. The differences in the resultant cognitive operation are determined by the anatomical and physiological properties of the transmodal node that acts as the critical gateway for the dominant transformation. Interconnected sets of transmodal nodes provide anatomical and computational epicentres for large-scale neurocognitive networks. In keeping with the principles of selectively distributed processing, each epicentre of a large-scale network displays a relative specialization for a specific behavioural component of its principal neurospychological domain. The destruction of transmodal epicentres causes global impairments such as multimodal anomia, neglect and amnesia, whereas their selective disconnection from relevant unimodal areas elicits modality-specific impairments such as prosopagnosia, pure word blindness and category-specific anomias. The human brain contains at least five anatomically distinct networks. The network for spatial awareness is based on transmodal epicentres in the posterior parietal cortex and the frontal eye fields; the language network on epicentres in Wernicke's and Broca's areas; the explicit memory/emotion network on epicentres in the hippocampal-entorhinal complex and the amygdala; the face-object recognition network on epicentres in the midtemporal and temporopolar cortices; and the working memory-executive function network on epicentres in the lateral prefrontal cortex and perhaps the posterior parietal cortex. Individual sensory modalities give rise to streams of processing directed to transmodal nodes belonging to each of these networks. The fidelity of sensory channels is actively protected through approximately four synaptic levels of sensory-fugal processing. The modality-specific cortices at these four synaptic levels encode the most veridical representations of experience. Attentional, motivational and emotional modulations, including those related to working memory, novelty-seeking and mental imagery, become increasingly more pronounced within downstream components of unimodal areas, where they help to create a highly edited subjective version of the world. (ABSTRACT TRUNCATED)

Theoretical Neuroanatomy: Relating Anatomical and Functional Connectivity in Graphs and Cortical Connection Matrices

O Sporns — 2008-01-07T04:44:34-00:00

Cereb. Cortex, Vol. 10, No. 2. (1 February 2000), pp. 127-141.

Neuroanatomy places critical constraints on the functional connectivity of the cerebral cortex. To analyze these constraints we have examined the relationship between structural features of networks (expressed as graphs) and the patterns of functional connectivity to which they give rise when implemented as dynamical systems. We selected among structurally varying graphs using as selective criteria a number of global information-theoretical measures that characterize functional connectivity. We selected graphs separately for increases in measures of entropy (capturing statistical independence of graph elements), integration (capturing their statistical dependence) and complexity (capturing the interplay between their functional segregation and integration). We found that dynamics with high complexity were supported by graphs whose units were organized into densely linked groups that were sparsely and reciprocally interconnected. Connection matrices based on actual neuroanatomical data describing areas and pathways of the macaque visual cortex and the cat cortex showed structural characteristics that coincided best with those of such complex graphs, revealing the presence of distinct but interconnected anatomical groupings of areas. Moreover, when implemented as dynamical systems, these cortical connection matrices generated functional connectivity with high complexity, characterized by the presence of highly coherent functional clusters. We also found that selection of graphs as they responded to input or produced output led to increases in the complexity of their dynamics. We hypothesize that adaptation to rich sensory environments and motor demands requires complex dynamics and that these dynamics are supported by neuroanatomical motifs that are characteristic of the cerebral cortex. 10.1093/cercor/10.2.127

Small worlds inside big brains

Olaf Sporns — 2007-01-10T14:06:01-00:00

PNAS, Vol. 103, No. 51. (19 December 2006), pp. 19219-19220.

10.1073/pnas.0609523103

The bifurcating neuron network 2: an analog associative memory

Geehyuk Lee — 2008-05-14T04:41:39-00:00

Neural Netw., Vol. 15, No. 1. (January 2002), pp. 69-84.

Synchronous neural activity in scale-free network models versus random network models

Geoffrey Grinstein — 2008-02-28T02:14:06-00:00

Proceedings of the National Academy of Sciences, Vol. 102, No. 28. (12 July 2005), pp. 9948-9953.

Synchronous firing peaks at levels greatly exceeding background activity have recently been reported in neocortical tissue. A small subset of neurons is dominant in a large fraction of the peaks. To investigate whether this striking behavior can emerge from a simple model, we constructed and studied a model neural network that uses a modified Hopfield-type dynamical rule. We find that networks having a power-law ("scale-free") node degree distribution readily generate extremely large synchronous firing peaks dominated by a small subset of nodes, whereas random (Erdos-Renyi) networks do not. This finding suggests that network topology may play an important role in determining the nature and magnitude of synchronous neural activity. 10.1073/pnas.0504127102

Dynamic Properties of Network Motifs Contribute to Biological Network Organization.

Robert J Prill — 2005-10-07T19:43:29-00:00

PLoS Biol, Vol. 3, No. 11. (4 October 2005)

Biological networks, such as those describing gene regulation, signal transduction, and neural synapses, are representations of large-scale dynamic systems. Discovery of organizing principles of biological networks can be enhanced by embracing the notion that there is a deep interplay between network structure and system dynamics. Recently, many structural characteristics of these non-random networks have been identified, but dynamical implications of the features have not been explored comprehensively. We demonstrate by exhaustive computational analysis that a dynamical property-stability or robustness to small perturbations-is highly correlated with the relative abundance of small subnetworks (network motifs) in several previously determined biological networks. We propose that robust dynamical stability is an influential property that can determine the non-random structure of biological networks.

Motifs in Brain Networks.

Olaf Sporns — 2004-11-22T00:17:30-00:00

PLoS Biol, Vol. 2, No. 11. (November 2004)

Complex brains have evolved a highly efficient network architecture whose structural connectivity is capable of generating a large repertoire of functional states. We detect characteristic network building blocks (structural and functional motifs) in neuroanatomical data sets and identify a small set of structural motifs that occur in significantly increased numbers. Our analysis suggests the hypothesis that brain networks maximize both the number and the diversity of functional motifs, while the repertoire of structural motifs remains small. Using functional motif number as a cost function in an optimization algorithm, we obtain network topologies that resemble real brain networks across a broad spectrum of structural measures, including small-world attributes. These results are consistent with the hypothesis that highly evolved neural architectures are organized to maximize functional repertoires and to support highly efficient integration of information.

Network structure of cerebral cortex shapes functional connectivity on multiple time scales

Christopher Honey — 2007-08-16T03:29:19-00:00

PNAS, Vol. 104, No. 24. (12 June 2007), pp. 10240-10245.

Neuronal dynamics unfolding within the cerebral cortex exhibit complex spatial and temporal patterns even in the absence of external input. Here we use a computational approach in an attempt to relate these features of spontaneous cortical dynamics to the underlying anatomical connectivity. Simulating nonlinear neuronal dynamics on a network that captures the large-scale interregional connections of macaque neocortex, and applying information theoretic measures to identify functional networks, we find structure-function relations at multiple temporal scales. Functional networks recovered from long windows of neural activity (minutes) largely overlap with the underlying structural network. As a result, hubs in these long-run functional networks correspond to structural hubs. In contrast, significant fluctuations in functional topology are observed across the sequence of networks recovered from consecutive shorter (seconds) time windows. The functional centrality of individual nodes varies across time as interregional couplings shift. Furthermore, the transient couplings between brain regions are coordinated in a manner that reveals the existence of two anticorrelated clusters. These clusters are linked by prefrontal and parietal regions that are hub nodes in the underlying structural network. At an even faster time scale (hundreds of milliseconds) we detect individual episodes of interregional phase-locking and find that slow variations in the statistics of these transient episodes, contingent on the underlying anatomical structure, produce the transfer entropy functional connectivity and simulated blood oxygenation level-dependent correlation patterns observed on slower time scales. 10.1073/pnas.0701519104

Reproducible Sequence Generation In Random Neural Ensembles

Ramón Huerta — 2008-03-20T09:52:59-00:00

Physical Review Letters, Vol. 93, No. 23. (2 December 2004), 238104.

Little is known about the conditions that neural circuits have to satisfy to generate reproducible sequences. Evidently; the genetic code cannot control all the details of the complex circuits in the brain. In this Letter; we give the conditions on the connectivity degree that lead to reproducible and robust sequences in a neural population of randomly coupled excitatory and inhibitory neurons. In contrast to the traditional theoretical view we show that the sequences do not need to be learned. In the framework proposed here just the averaged characteristics of the random circuits have to be under genetic control. We found that rhythmic sequences can be generated if random networks are in the vicinity of an excitatory-inhibitory synaptic balance. Reproducible transient sequences; on the other hand; are found far from a synaptic balance.

Dynamical Encoding by Networks of Competing Neuron Groups: Winnerless Competition

M Rabinovich — 2008-03-20T09:44:53-00:00

Physical Review Letters, Vol. 87, No. 6. (20 July 2001), 068102.

Following studies of olfactory processing in insects and fish; we investigate neural networks whose dynamics in phase space is represented by orbits near the heteroclinic connections between saddle regions (fixed points or limit cycles). These networks encode input information as trajectories along the heteroclinic connections. If there are N neurons in the network; the capacity is approximately e ( N -1)!; i.e.; much larger than that of most traditional network structures. We show that a small winnerless competition network composed of FitzHugh-Nagumo spiking neurons efficiently transforms input information into a spatiotemporal output.

Learning and approximation of chaotic time series using wavelet-networks

V Alarcon-Aquino — 2007-07-22T01:16:54-00:00

Computer Science, 2005. ENC 2005. Sixth Mexican International Conference on (2005), pp. 182-188.

This paper presents a wavelet neural-network for learning and approximation of chaotic time series. Wavelet networks are a class of neural network that take advantage of good localization and approximation properties of multiresolution analysis. These networks use wavelets as activation functions in the hidden layer and a hierarchical method is used for learning. Comparisons are made between a wavelet network, tested with two different wavelets, and the typical feedforward network trained with the back-propagation algorithm. The results reported in this paper show that wavelet networks have better approximation properties than back-propagation networks.

Improving Wavelet-Networks Performance with a New Correlation-based Initialisation Method and Training Algorithm

Edgar Garcia-Trevino — 2007-07-22T01:08:14-00:00

Computing, 2006. CIC '06. 15th International Conference on (2006), pp. 11-17.

Wavelet-networks are inspired by both the feedforward neural networks and the theory underlying wavelet decompositions. This special kind of networks has proved its advantages over other networks schemes, particularly in approximation and prediction problems. However, the training procedure used for wavelet networks is based on the idea of continuous differentiable wavelets, but unfortunately, most of powerful and used wavelets do not satisfy this property. This paper presents a new initialisation procedure and a new training algorithm for wavelet neural-networks that improve its performance allowing the use of different kind of wavelets. To show this, comparisons are made for chaotic time series approximation between the proposed approach and the typical wavelet-network

The Neural Coding of Stimulus Intensity: Linking the Population Response of Mechanoreceptive Afferents with Psychophysical Behavior

Michael Muniak — 2008-01-30T17:47:22-00:00

J. Neurosci., Vol. 27, No. 43. (24 October 2007), pp. 11687-11699.

How specific aspects of a stimulus are encoded at different stages of neural processing is a critical question in sensory neuroscience. In the present study, we investigated the neural code underlying the perception of stimulus intensity in the somatosensory system. We first characterized the responses of SA1 (slowly adapting type 1), RA (rapidly adapting), and PC (Pacinian) afferents of macaque monkeys to sinusoidal, diharmonic, and bandpass noise stimuli. We then had human subjects rate the perceived intensity of a subset of these stimuli. On the basis of these neurophysiological and psychophysical measurements, we evaluated a series of hypotheses about which aspect(s) of the neural activity evoked at the somatosensory periphery account for perception. We evaluated three types of neural codes. The first consisted of population codes based on the firing rate of neurons located directly under the probe. The second included population codes based on the firing rate of the entire population of active neurons. The third included codes based on the number of active afferents. We found that the response evoked in the localized population is logarithmic with stimulus amplitude (given a constant frequency composition), whereas the population response across all neurons is linear with stimulus amplitude. We conclude that stimulus intensity is best accounted for by the firing rate evoked in afferents located under or near the locus of stimulation, weighted by afferent type. 10.1523/JNEUROSCI.1486-07.2007

Efficient inhibition of bursts by bursts in the auditory system of crickets

Marsat — 2007-07-11T13:46:32-00:00

Journal of Comparative Physiology A, Vol. 193, No. 6. (June 2007), pp. 625-633.