torch.nn.parameter.Parameter

作用

a kind of Tensor that is to be considered a module parameter.

Parameter是一种可以作为模型参数的Tensor.

Parameters areTensorsubclasses, that have a very special property when used withModuleS ----when they’re assigned as Module attributes they are automatically added to the list of its parameters, and will appear e.g. inparameters()iterator. Assigning a Tensor doesn’t have such effect.

Parameter是Tensor的子类，同时拥有一种非常特殊的性质：当他们与Module S一起使用时，也就是说当它们作为Module参数进行使用时，它们会自动添加到Module的参数列表中，并且出现在parameters()迭代器里。(这样就可以自动计算梯度等)

构造参数

data(Tensor)-- parameter tensorrequires_grad(bool, optional)-- if the parameter requires gradient. Default: True.

Example

定义一个网络Module如下：

class LayerNorm(nn.Module):def __init__(self, features, eps=1e-6):super(LayerNorm, self).__init__()self.a_2 = nn.Parameter(torch.ones(features))self.b_2 = nn.Parameter(torch.zeros(features))self.eps = epsdef forward(self, x):mean = x.mean(-1, keepdim=True)std = x.std(-1, keepdim=True)return self.a_2 * (x - mean) / (std + self.eps) + self.b_2

那么，我们试着构造一个LayerNorm，来观察其参数：

>>> layerNorm = LayerNorm(5)>>> for a in layerNorm.parameters():print(a)Parameter containing:tensor([1., 1., 1., 1., 1.], requires_grad=True)Parameter containing:tensor([0., 0., 0., 0., 0.], requires_grad=True)

可以看到我们使用nn.Parameter进行构造的参数，自动传入了Module的参数列表。