gelu

paddle.nn.functional. gelu ( x: Tensor, approximate: Literal['tanh', 'none'] | bool = False, name: str | None = None ) → Tensor [source]

gelu activation.

The activation function of Gelu is calculated element by element. More information refers to :ref: Gaussian Error Linear Units.

approximate parameter must be True, False, “tanh”, “none”.

if approximate is True or “tanh”

\[gelu(x) = 0.5 * x * (1 + tanh(\sqrt{\frac{2}{\pi}} * (x + 0.044715x^{3})))\]

else

\[gelu(x) = 0.5 * x * (1 + erf(\frac{x}{\sqrt{2}}))\]

Parameters

• x (Tensor) – The input Tensor with data type float32, float64.

• approximate (str|bool, optional) – Whether to enable approximation. Default is False.

• name (str|None, optional) – For details, please refer to api_guide_Name. Generally, no setting is required. Default: None.

Returns

A Tensor with the same data type and shape as x .

Examples

>>> import paddle
>>> import paddle.nn.functional as F

>>> x = paddle.to_tensor([[-1, 0.5], [1, 1.5]])
>>> out1 = F.gelu(x)
>>> print(out1)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15865529,  0.34573123],
 [ 0.84134471,  1.39978933]])
>>> out2 = F.gelu(x, True)
>>> print(out2)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15880796,  0.34571400],
 [ 0.84119201,  1.39957154]])
>>> out3 = F.gelu(x, "none")
>>> print(out3)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15865529,  0.34573123],
 [ 0.84134471,  1.39978933]])
>>> out4 = F.gelu(x, "tanh")
>>> print(out4)
Tensor(shape=[2, 2], dtype=float32, place=Place(cpu), stop_gradient=True,
[[-0.15880796,  0.34571400],
 [ 0.84119201,  1.39957154]])