DL学习笔记【20】nn包中的各位Simple layers

本文主要是介绍DL学习笔记【20】nn包中的各位Simple layers，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！

y = Ax + b

x = torch.Tensor({ {1, 0.1}, {2, 0.3}, {10, 0.3}, {31, 0.2} })print(x)1.0000   0.10002.0000   0.300010.0000   0.300031.0000   0.2000

\forall k: y_k = x_1 A_k x_2 + b

 input = {torch.randn(128, 10), torch.randn(128, 5)}  -- 128 input examplesmodule:forward(input)

module = nn.PartialLinear(5, 3)  -- 5 inputs, 3 outputs
module:setPartition(torch.Tensor({2,4})) -- only compute the 2nd and 4th indices out of a total of 5 indices

y_j = || w_j - x ||

y_j = || c_j * (w_j - x) ||

y_j = (x · w_j) / ( || w_j || * || x || )

pred_mlp = nn.Sequential()  -- A network that makes predictions given x.
pred_mlp:add(nn.Linear(5, 4))
pred_mlp:add(nn.Linear(4, 3))xy_mlp = nn.ParallelTable() -- A network for predictions and for keeping the
xy_mlp:add(pred_mlp)        -- true label for comparison with a criterion
xy_mlp:add(nn.Identity())   -- by forwarding both x and y through the network.mlp = nn.Sequential()       -- The main network that takes both x and y.
mlp:add(xy_mlp)             -- It feeds x and y to parallel networks;
cr = nn.MSECriterion()
cr_wrap = nn.CriterionTable(cr)
mlp:add(cr_wrap)            -- and then applies the criterion.for i = 1, 100 do           -- Do a few training iterationsx = torch.ones(5)        -- Make input features.y = torch.Tensor(3)y:copy(x:narrow(1,1,3))  -- Make output label.err = mlp:forward{x,y}   -- Forward both input and output.print(err)               -- Print error from criterion.mlp:zeroGradParameters() -- Do backprop...mlp:backward({x, y})mlp:updateParameters(0.05)
end

Modules that adapt basic Tensor methods :

Copy :a copy of the input with type casting ; 看解释好像是把input复制到output中，不太了解用处嗯。。output和input的值一样多么？

Narrow : a narrow operation over a given dimension ;

Replicate : repeats input n times along its first dimension ;

Reshape : a reshape of the inputs ;

View : a view of the inputs ;

Contiguous : contiguous of the inputs ;

Select : a select over a given dimension ;

MaskedSelect : a masked select module performs the torch.maskedSelect operation ;

Index : a index over a given dimension ;

Squeeze : squeezes the input;

Unsqueeze : unsqueeze the input, i.e., insert singleton dimension;

Transpose : transposes the input ;

Modules that adapt mathematical Tensor methods :

AddConstant : adding a constant ;

MulConstant : multiplying a constant ;

Max : a max operation over a given dimension ;

Min : a min operation over a given dimension ;

Mean : a mean operation over a given dimension ;

Sum : a sum operation over a given dimension ;

Exp : an element-wise exp operation ;

Log : an element-wise log operation ;

Abs : an element-wise abs operation ;

Power : an element-wise pow operation ;

Square : an element-wise square operation ;

Sqrt : an element-wise sqrt operation ;

Clamp : an element-wise clamp operation ;

Normalize : normalizes the input to have unit L_p norm ;

MM : matrix-matrix multiplication (also supports batches of matrices) ;

Miscellaneous Modules :

BatchNormalization : mean/std normalization over the mini-batch inputs (with an optional affine transform) ;

PixelShuffle : Rearranges elements in a tensor of shape [C*r, H, W] to a tensor of shape [C, H*r, W*r] ;

Identity : forward input as-is to output (useful with ParallelTable) ;

Dropout : masks parts of the input using binary samples from a bernoulli distribution ;

SpatialDropout : same as Dropout but for spatial inputs where adjacent pixels are strongly correlated ;

VolumetricDropout : same as Dropout but for volumetric inputs where adjacent voxels are strongly correlated ;

Padding : adds padding to a dimension ;

L1Penalty : adds an L1 penalty to an input (for sparsity) ;

GradientReversal : reverses the gradient (to maximize an objective function) ;

GPU : decorates a module so that it can be executed on a specific GPU device.

TemporalDynamicKMaxPooling : selects the k highest values in a sequence. k can be calculated based on sequence length ;

这篇关于DL学习笔记【20】nn包中的各位Simple layers的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！