rms_norm

paddle.nn.functional. rms_norm ( input: Tensor, normalized_shape: Sequence[int], weight: Tensor | None = None, eps: float | None = None, name: str | None = None ) → Tensor [source]

Applies Layer Normalization over the last dimension of the input tensor using CUDA implementation.

Parameters

  • input (Tensor) – Input tensor of shape [rows, cols] or higher dimensions (flattened to 2D).

  • normalized_shape (list|tuple) – Input shape from an expected input of size \([*, normalized_shape[0], normalized_shape[1], ..., normalized_shape[-1]]\). If it is a single integer, this module will normalize over the last dimension which is expected to be of that specific size.

  • weight (Tensor, optional) – The weight tensor of rms_norm. Default: None.

  • eps (float|None, optional) – The small value added to the variance to prevent division by zero. If None, uses machine epsilon for the compute dtype: float64 inputs use np.finfo(np.float64).eps (double epsilon), all other dtypes use np.finfo(np.float32).eps (float epsilon). Default: None.

  • name (str, optional) – Name of the operator.

Returns

Normalized tensor of same shape as input.

Return type

out (Tensor)