Neural networks

What is a neural network?
Neural Networks are a different paradigm for computing:
von Neumann machines are based on the processing/memory abstraction of human information processing.
neural networks are based on the parallel architecture of animal brains.
Neural networks are a form of multiprocessor computer system, with
simple processing elements
a high degree of interconnection
simple scalar messages
adaptive interaction between elements
A biological neuron may have as many as 10,000 different inputs, and may send its output (the presence or absence of a short-duration spike) to many other neurons. Neurons are wired up in a 3-dimensional pattern.
Running the network consists of
Forward pass:
the outputs are calculated and the error at the output units calculated.
Backward pass:
The output unit error is used to alter weights on the output units. Then the error at the hidden nodes is calculated (by back-propagating the error at the output units through the weights), and the weights on the hidden nodes altered using these values.
For each data pair to be learned a forward pass and backwards pass is performed. This is repeated over and over again until the error is at a low enough level (or we give up).









Radial Basis function Networks
Radial basis function networks are also feedforward, but have only one hidden layer.
Typical RBF architecture:
Like BP, RBF nets can learn arbitrary mappings: the primary difference is in the hidden layer.
RBF hidden layer units have a receptive field which has a centre: that is, a particular input value at which they have a maximal output.Their output tails off as the input moves away from this point.
Generally, the hidden unit function is a Gaussian:
Gaussians with three different standard deviations.
Training RBF Networks.
RBF networks are trained by
deciding on how many hidden units there should be
deciding on their centres and the sharpnesses (standard deviation) of their Gaussians
training up the output layer.
Generally, the centres and SDs are decided on first by examining the vectors in the training data. The output layer weights are then trained using the Delta rule. BP is the most widely applied neural network technique. RBFs are gaining in popularity.
Nets can be
trained on classification data (each output represents one class), and then used directly as classifiers of new data.
trained on (x,f(x)) points of an unknown function f, and then used to interpolate.
RBFs have the advantage that one can add extra units with centres near parts of the input which are difficult to classify. Both BP and RBFs can also be used for processing time- varying data: one can consider a window on the data:
Networks of this form (finite-impulse response) have been used in many applications.
There are also networks whose architectures are specialised for processing time-series.
Unsupervised networks:
Simple Perceptrons, BP, and RBF networks need a teacher to tell the network what the desired output should be. These are supervised networks.
In an unsupervised net, the network adapts purely in response to its inputs. Such networks can learn to pick out structure in their input.
Applications for unsupervised nets
clustering data:
exactly one of a small number of output units comes on in response to an input.
reducing the dimensionality of data:
data with high dimension (a large number of input units) is compressed into a lower dimension (small number of output units).
Although learning in these nets can be slow, running the trained net is very fast - even on a computer simulation of a neural net.
Kohonen clustering Algorithm:
- takes a high-dimensional input, and clusters it, but retaining some topological ordering of the output.
After training, an input will cause some the output units in some area to become active.
Such clustering (and dimensionality reduction) is very useful as a preprocessing stage, whether for further neural network data processing, or for more traditional techniques.
Where are Neural Networks applicable?
..... or are they just a solution in search of a problem?
Neural networks cannot do anything that cannot be done using traditional computing techniques, BUT they can do some things which would otherwise be very difficult.
In particular, they can form a model from their training data (or possibly input data) alone.
This is particularly useful with sensory data, or with data from a complex (e.g. chemical, manufacturing, or commercial) process. There may be an algorithm, but it is not known, or has too many variables. It is easier to let the network learn from examples.
Neural networks are being used:
in investment analysis:
to attempt to predict the movement of stocks currencies etc., from previous data. There, they are replacing earlier simpler linear models.
in signature analysis:
as a mechanism for comparing signatures made (e.g. in a bank) with those stored. This is one of the first large-scale applications of neural networks in the USA, and is also one of the first to use a neural network chip.
in process control:
there are clearly applications to be made here: most processes cannot be determined as computable algorithms. Newcastle University Chemical Engineering Department is working with industrial partners (such as Zeneca and BP) in this area.
in monitoring:
networks have been used to monitor
· the state of aircraft engines. By monitoring vibration levels and sound, early warning of engine problems can be given.
· British Rail have also been testing a similar application monitoring diesel engines.

in marketing:
networks have been used to improve marketing mailshots. One technique is to run a test mailshot, and look at the pattern of returns from this. The idea is to find a predictive mapping from the data known about the clients to how they have responded. This mapping is then used to direct further mailshots.