Function deep_learning_playground::neural_network::activate_functions::softmax

pub fn softmax<T: Float>() -> Box<dyn Fn(Array2<T>) -> Array2<T>>

softmax generates the softmax function: \[ \text{SoftMax}(\boldsymbol{x})=\left( \dfrac{\exp(x_1)}{\displaystyle\sum^n_{j=1}\exp(x_j)}, \dfrac{\exp(x_2)}{\displaystyle\sum^n_{j=1}\exp(x_j)}, \cdots, \dfrac{\exp(x_n)}{\displaystyle\sum^n_{j=1}\exp(x_j)} \right) \] for Array2 where \(\boldsymbol{x}=\left(x_1,\cdots,x_n\right)^T\subseteq\mathbb{R}^{n\times 1}\). To prevent overflow, actually calculate according to the following equation: \[ \begin{array}{lll} \dfrac{\exp(x_i)}{\displaystyle\sum^n_{j=1}\exp(x_j)}&=&\dfrac{C\exp(x_i)}{C\displaystyle\sum^n_{j=1}\exp(x_j)}\\ &=&\dfrac{\exp(x_i+\log C)}{\displaystyle\sum^n_{j=1}\exp(x_j+\log C)}\\ &=&\dfrac{\exp(x_i+C')}{\displaystyle\sum^n_{j=1}\exp(x_j+C')} \end{array} \] Therefore, \(C'\) can be for all value. Thus \(C'=x_{\text{max}}\) where \(^\forall x_i, ^\exists x_{\text{max}}\in\boldsymbol{x}\) s.t. \(x_{\text{max}}\geq x_i\).