Back-propagation is not (in my opinion) a concept easily grasped without some hands-on experience. Therefore, if you are a programmer, first look at the code I wrote, then experiment with a BP network of your own.
Back-propagation A method for maximum likelihood estimation of a feed-forward neural network. It is equivalent to steepest-descent optimization. See Bishop.
A back-propagation ANN, conversely, is trained by humans to perform specific tasks. During the training period, the teacher evaluates whether the ANN's output is correct.
Static back-propagation is one kind of backpropagation networks that produces a mapping of a static input to a static output. These networks can solve static classification problems such as optical character recognition (OCR).
5.1.1 Feedforward, Back-Propagation.
5.1.2 Delta Bar Delta.
5.1.3 Extended Delta Bar Delta.
5.1.4 Directed Random Search.
5.1.5 Higher-order Neural Network or Functional-link Network.
5.1.6 Self-Organizing Map into Back-Propagation.
Is there a way of understanding back-propagation other than reciting the necessary equations? The answer is, of course, yes. In fact, the whole back-propagation process is intuitively very clear. What happens in the above equations is the following.
4 The Back-Propagation Algorithm 129
4.5 XOR Problem 141
4.6 Heuristics for Making the Back-Propagation Algorithm Perform Better 144
4.7 Computer Experiment: Pattern Classification 150
4.8 Back Propagation and Differentiation 153
4.
For instance, multi-layer perceptron back-propagation can be substituted with more general global optimization techniques. The objective in training a ANN is, given some set of pairs of data and output, { (d0, o0) , (d1,o1), ...
The Super self-adjusting back-propagation algorithm (SuperSAB) was developed by Tom Tollenaere.
Training a single layer requires no back-propagation of error and can be done very efficiently. At some point further training will not produce much improvement. If network performance is satisfactory, training can be stopped.
After running the back-propagation learning algorithm on a given set of examples, the neural network can be used to predict outcomes for any set of input values.
"The most popular method for learning in multilayer networks is called Back-propagation. It was first invented in 1969 by Bryson and Ho, but was more or less ignored until the mid-1980s."
^ Arthur Earl Bryson, Yu-Chi Ho (1969).
Werbos invented the back-propagation algorithm, that enabled multilayer neural networks, that had the ability to perform classification operations beyond simple Perceptrons.
An example is the Multi-Layer Perceptron trained with the "back-propagation" algorithm.
Fahlman, S.E. (1988). Faster-learning variations on back-propagation: an empirical study. In D. Touretzky, G.E. Hinton and T.J. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Models Summer School, 38-51. San Mateo, CA: Morgan Kaufmann.
Using back-propagation requires only enough memory for the network itself, as well as loop variables, etc. The genetic algorithm requires a population of networks (the larger, the better my population of 5 is pretty small).
The learning rate also depends, of course, on the algorithm implemented. A chip with a RBF algorithm could have a slower learning pass than a feed-forward chip trained with back-propagation, but learns with far fewer passes.
have become increasingly popular in feature recognition and function mapping (approximation) problems in a wide area of applications.
Because of commonly used learning algorithm this type of networks are sometimes referred to as back-propagation ...
sigmoid S-shaped. In neural networks, the use of a sigmoid transfer function (as opposed to a step function) allows the function to be differentiated and thus allows back-propagation of values for automated learning.
Therefore, the popular rumor is wrong: that Back-Propagation remedies this, because no matter how fast such a machine can learn, it can't find solutions that don't exist.
(For those interested the technical name for this procedure is called back-propagation). This process is somewhat similar to the way a memory is recalled in the Hopfield network. We can imagine a ball rolling on surface.
 See also: Neural network, Perceptron, Percept, Classification, Artificial neural network
  
|
RSS
