megnet.activations module

Activation functions used in neural networks

get(identifier: Optional[Union[str, Callable[[...], Any]]] = None) → Callable[[...], Any]

Get activations by identifier

Parameters

identifier (str or callable) – the identifier of activations

Returns

callable activation

softplus2(x)

out = log(exp(x)+1) - log(2) softplus function that is 0 at x=0, the implementation aims at avoiding overflow

Parameters

x – (Tensor) input tensor

Returns

(Tensor) output tensor

swish(x)

out = x * sigmoid(x)

Parameters

x – (Tensor) input tensor

Returns

(Tensor) output tensor