It is one of the most used activation functions in DNN. The major advantage behind this activation function is that it does not trigger all the neurons at the same time and converts all the negative input into zero so that the neuron does not get activated. Training performance in this activation function is much faster as compared with other activation functions (Haykin, 1994). It can be represented asf(x)=max(0,x),(13.5)
where x is the input data and f(x) the output function that returns the maximum value between 0 and input data.
Leave a Reply