当激活功能层的输入值为最大值或最小值时，容易出现梯度消失。ReLU激活

When the input value of the activation function layer is the maximum or the minimum, the gradient disappears easily. The ReLU activation function can maintain the original gradient when the input value is greater than 0, and alleviate the line image with non-gradient disappearance. It is currently the most mainstream activation function, but when its input value is less than 0, it will be suppressed and the negative gradient disappears. .

When the input value of the activation function layer is the maximum or minimum value, it is easy to see the gradient disappear. ReLU activation function can maintain the original gradient when the input value is greater than 0, alleviate the non-gradient disappearing line image, is currently the most mainstream activation function, but when its input value is less than 0, will be suppressed, there is a negative gradient disappearing phenomenon.

When the input value of the active functional layer is the maximum or the minimum, the gradient is easy to disappear. The relu activation function can keep the original gradient when the input value is greater than 0, and alleviate the line image which is not gradient disappearing. It is the most popular activation function at present, but when the input value is less than 0, it will be suppressed, and there is a phenomenon of negative gradient disappearing.<br>