python - 参数的光谱范数如何计算？(How is spectral norm of a parameter calculated?)

Question

when I do,

(当我做，)

import torch, torch.nn as nn
x = nn.Linear(3, 3)
y = torch.nn.utils.spectral_norm(x)

then it gives four different weight matrices,

(然后给出四个不同的权重矩阵)

y.weight_u

tensor([ 0.6534, -0.1644,  0.7390])

y.weight_orig

Parameter containing:
tensor([[ 0.2538,  0.3196,  0.3380],
        [ 0.4946,  0.0519,  0.1022],
        [-0.5549, -0.0401,  0.1654]], requires_grad=True)

y.weight_v

tensor([-0.3650,  0.2870,  0.8857])

y.weight

tensor([[ 0.5556,  0.6997,  0.7399],
        [ 1.0827,  0.1137,  0.2237],
        [-1.2149, -0.0878,  0.3622]], grad_fn=<DivBackward0>)

how are these four matrices calculated?

(如何计算这四个矩阵？)

  ask by vainaijr translate from so

Answer 1

等待大神答复