## Modeling biological neurons

Neural nets are modeled upon real biological neurons, which have the following characteristics:

**All or none**: the input from each entry in the neural net is either O or 1, the output is also 0 or 1**Cumulative influence**: the influence of various input neurons accumulates to produce the final output result, if a certain threshold is reached**Synaptic weight**: each input neuron is given a weight depending on its importance

### Characteristics that are not modeled

Several characteristics of biological neurons are not modeled in neural nets, as their purpose is not clearly understood in brain information processing.

- Real neurons also have a
**refractory period**when they do not respond to stimuli after firing. - In a real neuron, the resulting output can go to either one (of a few)
**axonal bifurcations**. - The
**timing of the various inputs**in the dendritic tree is not well understood in the resulting pulse in the axon.

## Neural net principles

In a neural net, the resulting vector *z*, is a function of the different inputs *x[n]*, the weights *w[n]* and the thresholds *t[n]*. A neural net is therefore a **function approximator**.

By comparing a known desired results vector (such as the content of a picture) with the actual output vector, a **performance function** can be determined to know how well the neural net is performing.

To simplify the performance function, thresholds are eliminated by adding an **extra weight w[0]** that nullify the threshold, and the step function resulting in {0, 1} values is smoothed to a

**sigmoid function**resulting in the [0, 1] interval.

### Backpropagation

Backpropagation is the name of the algorithm generally used to **train a neural net**.

**Varying the weights little by little and with a certain randomization** allows the performance function to measure if progress is being made or not, and to improve the weighing accordingly to **progress towards an optimal performance**.

The amount of computation of the performance function is linearly increased by the depth and squared by the width of the net.