fused_rms_norm_ext

paddle.incubate.nn.functional. fused_rms_norm_ext ( x, scale, epsilon=1e-05, name=None ) [source]

Applies Layer Normalization over the last dimension of the input tensor using CUDA implementation. :param x: Input tensor of shape [rows, cols] or higher dimensions (flattened to 2D). :type x: Tensor :param scale: Scale tensor of shape [cols]. :type scale: Tensor :param bias: Bias tensor of shape [cols]. If None, no bias is added. :type bias: Tensor, optional :param epsilon: Small constant to avoid division by zero. :type epsilon: float :param name: Name of the operator. :type name: str, optional

Returns

Normalized tensor of same shape as x. mean (Tensor): Tensor of shape [rows], the mean of each row. invvar (Tensor): Tensor of shape [rows], the inverse standard deviation of each row.

Return type

y (Tensor)