Introduction to Neural Networks

Course content

Summary

We then introduce one kind of network in detail: the feedforward network trained by backpropagation of error. We discuss model architectures, training methods and data representation issues. We hope to cover everything you need to know to get backpropagation working for you. A range of applications and extensions to the basic model will be presented in the final section of the module.

Lecture 1: Introduction

• Computation in the brain
• Artificial neuron models
• Linear regression
• Linear neural networks
• Multi-layer networks
• Error Backpropagation

• Revision: the backprop algorithm
• Backprop: an example
• Overfitting and regularization
• Growing and pruning networks
• Preconditioning the network
• Momentum and learning rate adaptation

• Recurrent Neural Networks
• Learning Rate Adaptation
• Tasks and error functions:
  • Classification
  • Autoassociation and prediction
  • Reinforcement Learning

By popular demand: the whole course (well, Lectures 1 & 2 so far) as a ZIP file.

Organization

Contact

• Email Nici
• Email Fred

Links

• The Nervous System - a very nice introduction, many pictures

• Neural Java - a neural network tutorial with Java applets

• a book chapter describing the Backpropagation Algorithm (Postscript)

• A short set of pages showing how a simple backprop net learns to recognize the digits 0-9, with C code

	Brainwave: a Java based simulator
	tlearn: W*ndows, M*cintosh and Un*x implentation of backprop and variants. Written in C.
	PDP++: C++ software with every conceivable bell and whistle. Un*x only. The manual also makes a good tutorial.

Related stuff of interest:

• A page of neural network links

• Reinforcement Learning - A Tutorial