A sigmoid function is a mathematical function having a characteristic “S”-shaped curve or sigmoid curve. It is used in neural networks as an activation function, defining the output of a node given a set of inputs.
The Sigmoid function often refers to the special case of the logistic function shown in the first figure and defined by the formula:
The graph representing this function forms an “S” shape from R to ]0;1[
The sigmoid function has a first derivative that is always non-negative and forms a bell shape.
Read more on the sigmoid function on Wikipedia.
The use of the sigmoid function in neural networks
The sigmoid function is used to program neural networks as an activation function. It is a standard function in a neural net node (a “neuron”), used to normalize the sum of data inputs after applying weights.
It ensures the output of the node is always in the interval ]0;1[, while also signifying high confidence for large positive or negative numbers.
In practice the sigmoid function used in neural nets has the following formula:
The sigmoid function derivative is used in the training period to modify weights applied to the training data, during the process of gradient descent.
Read more on the use of the sigmoid function in the post on programming a simple neural network or review the video on principles of neural nets.
« Back to Glossary Index