Switch Activation Function

The switch activation function is defined as:

Usage

Swish(x)

Arguments

x

A numeric vector. All elements must be finite and non-missing.

Value

A numeric vector where the ReLU function has been applied to each element of x.

Details

$$f(x) = x \cdot \text{sigmoid}(x)$$

See also

Other Activation Functions: ELU(), GELU(), PReLU(), ReLU(), SELU(), dELU(), dGELU(), dPReLU(), dReLU(), dSELU(), dSwish(), dlReLU(), dsoftplus(), lReLU(), softplus()

Author

Matthew Fidler

Examples

Swish(c(-1, 0, 1, 2))
#> [1] -0.2689414  0.0000000  0.7310586  1.7615942

# Can also be used in rxode2:
x <- rxode2({
   r<- Swish(time)
})
#>  
#>  
#> using C compiler: ‘gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0’

e <- et(c(-1, 0, 1, 2))

rxSolve(x, e)
#> ── Solved rxode2 object ──
#> ── Parameters (value$params): ──
#> # A tibble: 1 × 0
#> ── Initial Conditions (value$inits): ──
#> named numeric(0)
#> ── First part of data (object): ──
#> # A tibble: 4 × 2
#>    time      r
#>   <dbl>  <dbl>
#> 1    -1 -0.269
#> 2     0  0    
#> 3     1  0.731
#> 4     2  1.76