Sigmoid function

Whether you put into effect a neural community yourself or you operate a constructed library for neural community getting to know, it’s miles of paramount significance to recognize the importance of a sigmoid function. The sigmoid function is an important thing to expertise how a neural community learns complicated problems. This characteristic additionally served as a foundation for coming across different capabilities that cause green and desirable answers for supervised getting-to-know in deep getting-to-know architectures.

In this tutorial, you may find out the sigmoid function and its function in getting to know from examples in the neural community

Sigmoid Function

The sigmoid characteristic is a unique shape of the logistic characteristic and is normally denoted via way of means of σ(x) or sin(x). It is given via way of means of:

σ(x) = 1/(1+exp(-x))

What is the Sigmoid Function?

A Sigmoid characteristic is a mathematical characteristic that has a feature S-formed curve. There are some of the not unusual place sigmoid capabilities, which include the logistic characteristic, the hyperbolic tangent, and the arctangent

In device getting to know, the term

sigmoid characteristic is commonly used to refer especially to the logistic characteristic, additionally referred to as the logistic sigmoid characteristic.

All sigmoid capabilities have the assets that they map the complete wide variety line right into a small variety which includes zero and 1, or -1 and 1, so one use of a sigmoid characteristic is to transform an actual fee into one which may be interpreted as a probability.

Properties and Identities Of Sigmoid Function

The graph of sigmoid characteristic is an S-formed curve as proven via way of means of the inexperienced line withinside the graph underneath. The parent additionally suggests the graph of the spinoff in crimson color. The expression for the spinoff, together with a few critical houses is proven at the proper.

A few different houses include:

  1. Domain: (-∞, +∞)
  2. Range: (zero, +1)
  3. σ(zero) = zero.5
  4. The characteristic is monotonically increasing.
  5. The characteristic is non-stop anywhere.
  6. The characteristic is differentiable anywhere in its area.
  7. Numerically, it’s miles sufficient to compute this characteristic’s fee over a small variety of numbers, e.g., [-10, +10]. For values much less than -10, the characteristic’s fee is sort of zero. For values extra than 10, the characteristic’s values are nearly one.

The Sigmoid As A Squashing Function

The sigmoid characteristic is likewise referred to as a squashing characteristic as its area is the set of all actual numbers, and its variety is (zero, 1). Hence, if the entry to the character is both a completely big bad wide variety or a completely big high-quality wide variety,  the output is continually between zero and 1. The same is going for any wide variety among -∞ and +∞.

Sigmoid As an Activation Function in Neural Networks in system learning

The sigmoid characteristic is used as an activation characteristic in neural networks. Just to check what’s an activation characteristic, the parent underneath suggests the function of an activation characteristic in a single layer of a neural community. A weighted sum of inputs is surpassed via an activation characteristic and this output serves as an entry to the subsequent layer. 

When the activation characteristic for a neuron is sigmoid it’s miles a assure that the output of this unit will continually be between zero and 1. Also, because the sigmoid is a non-linear characteristic, the output of this unit could be a non-linear characteristic of the weighted sum of inputs. Such a neuron that employs a sigmoid characteristic as an activation characteristic is called a sigmoid unit.

Linear Vs. Non-Linear Separability?

Suppose we have an average class trouble, wherein we have a fixed of factors in the area and every factor is assigned a category label. If an instant line (or a hyperplane in an n-dimensional area) can divide the 2 classes, then we have linearly separable trouble. On the alternative hand, if an instant line isn’t always sufficient to divide the 2 classes, then we have non-linearly separable trouble. The parent underneath suggests facts withinside the 2-dimensional area. Each factor is assigned a pink or blue elegance label. The left parent suggests a linearly separable trouble that calls for a linear boundary to differentiate between the 2 classes. The proper parent suggests non-linearly separable trouble, wherein a non-linear selection boundary is required.

For 3 dimensional area, a linear selection boundary may be defined thru the equation of a plane. For an n-dimensional area, the linear selection boundary is defined via way of means of the equation of a hyperplane.

Why The Sigmoid Function Is Important In Neural Networks?

If we use a linear activation characteristic in a neural community, then this version can simplest study linearly separable problems. However, with the addition of simply one hidden layer and a sigmoid activation characteristic withinside the hidden layer, the neural community can without difficulty study non-linearly separable trouble. Using a non-linear characteristic produces non-linear barriers and hence, the sigmoid characteristic may be utilized in neural networks for getting to know complicated selection capabilities.

The simplest non-linear characteristic that may be used as an activation characteristic in a neural community is one that is monotonically increasing. So for example, sin(x) or cos(x) can not be used as activation capabilities. Also, the activation characteristic must be described anywhere and must be non-stop anywhere withinside the area of actual numbers. The characteristic is likewise required to be differentiable over the complete area of actual numbers.

Typically a returned propagation set of rules makes use of gradient descent to study the weights of a neural community. To derive this set of rules, the spinoff of the activation characteristic is required.

The truth that the sigmoid characteristic is monotonic, non-stop, and differentiable anywhere, coupled with the assets that its spinoff may be expressed in phrases of itself, makes it smooth to derive the replace equations for getting to know the weights in a neural community whilst the use of returned propagation set of rules.



In this tutorial, you located what’s a sigmoid function. Specifically, you learned:

  • The sigmoid characteristic and its houses
  • Linear vs. non-linear selection barriers
  • Why does including a sigmoid characteristic on the hidden layer permit a neural community to study complicated non-linear barriers

Do you’ve got any questions?

Ask your questions at and I will do my first-rate to answer

Leave a Comment