pytorch文档阅读（一）

本章主要针对pytorch0.4.0英文文档的前两节,顺序可能有些不一样：

torch
torch.Tensor

张量 Tensors

Data type	CPU tensor	GPU tensor	type
32-bit floating point	torch.FloatTensor	torch.cuda.FloatTensor	torch.float32
64-bit floating point	torch.DoubleTensor	torch.cuda.DoubleTensor	torch.float64
16-bit floating point	N/A	torch.cuda.HalfTensor	torch.cuda.float16
8-bit integer (unsigned)	torch.ByteTensor	torch.cuda.ByteTensor	torch.uint8
8-bit integer (signed)	torch.CharTensor	torch.cuda.CharTensor	torch.int8
16-bit integer (signed)	torch.ShortTensor	torch.cuda.ShortTensor	torch.int16
32-bit integer (signed)	torch.IntTensor	torch.cuda.IntTensor	torch.int32
64-bit integer (signed)	torch.LongTensor	torch.cuda.LongTensor	torch.int64

torch.is_stensor/torch.is_storage
torch.set_default_tensor_type()
这个有用，如果大部分操作是GPU上构建的，可你把默认类型定为cuda tensor

if torch.cuda.is_available():
    if args.cuda:
        torch.set_default_tensor_type('torch.cuda.FloatTensor')
    if not args.cuda:
        print("WARNING: It looks like you have a CUDA device, but aren't " +
              "using CUDA.\nRun with --cuda for optimal training speed.")
        torch.set_default_tensor_type('torch.FloatTensor')
else:
    torch.set_default_tensor_type('torch.FloatTensor')

torch.numel(input)->int/numel() /nelement()
torch.set_printoptions(precision=None, threshold=None, edgeitems=None, linewidth=None, profile=None) 打印选项
torch.set_flush_denormal(mode) → bool 禁用cpu非常规浮点

>>> torch.set_flush_denormal(True)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor([ 0.], dtype=torch.float64)
>>> torch.set_flush_denormal(False)
True
>>> torch.tensor([1e-323], dtype=torch.float64)
tensor(9.88131e-324 *
       [ 1.0000], dtype=torch.float64)

## 创建操作 Creation Ops

torch方法后缀_like :创建除了值以外的任何设置相同的tensor
包括：zeros，ones，empty， full，rand，randint，randn

torch.tensor(data, dtype=None, device=None, requires_grad=False) → Tensor
torch.from_numpy(ndarray) → Tensor
torch.eye(n, m=None, out=None)
torch.linspace(start, end, steps=100, out=None) → Tensor
torch.logspace(start, end, steps=100, out=None) → Tensor
torch.ones(*sizes, out=None) → Tensor
torch.empty(*sizes, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.reshape(input, shape) → Tensor 注意这是个坑，
torch.rand(*sizes, out=None) → Tensor（均匀分布）
torch.randn(*sizes, out=None) → Tensor（正态分布）
torch.randperm(n, out=None) → LongTensor（随机整数0，，，n-1）
torch.randint(low=0, high, size, out=None, dtype=None, layout=torch.strided, device=None, requires_grad=False) → Tensor
torch.arange(start, end, step=1, out=None) → Tensor/torch.range(start, end, step=1, out=None) → Tensor 一个包含末尾，一个不包含
torch.zeros(*sizes, out=None) → Tensor

## 索引,切片,连接,编译操作

这些操作绝大多数tensor本身也包含
tensor方法的通用后缀 _ inplace操作，

torch.cat(seq, dim=0, out=None) → Tensor/torch.stack(sequence, dim=0) 常用操作,一个是存在的维度上，一个是新的维度上（新建一个维度，已经存在的维度自然向后挤了）
torch.split(tensor, split_size, dim=0)/torch.chunk(tensor, chunks, dim=0)/split()/chunk(),这两个功能相近，一个是沿轴均分指定大小（如果无法整除，最后一块返回较小的块），另一个chunk是返回固定块数（也是和split一样，最后一块返回较小块）

a = torch.Tensor([1,2,3,4,5])
b = a.split(2)
c = a.chunk(3)

torch.gather(input, dim, index, out=None) → Tensor/gather(dim, index) 这个函数就很迷了，当时学习tensorflow时就研究了好久╮(╯﹏╰）╭，注意所有的index都是torch.LongTensor

 torch.gather(t, 1, torch.LongTensor([[0,0],[1,0]]))

torch.index_select(input, dim, index, out=None) → Tensor/index_select(dim, index) f非常重要的函数
torch.masked_select(input, mask, out=None) → Tensor/masked_select(mask) 注意所有mask都为torch.ByteTensor,同时需要注意mask的shape不一定要和tensor相同数量也不一定要相同，shape中必须有一个轴要和tensor的轴对应，此时按此轴索引

a = torch.Tensor([[1,2,3],[4,5,6]])
mask = torch.Tensor([[1,0],[0,0],[1,0]]).type(torch.ByteTensor)
mask_1 = torch.Tensor([[1],[0]]).type(torch.ByteTensor)
mask_2 = torch.Tensor([0,1,1]).type(torch.ByteTensor)
b = a.masked_select(mask)#error
c = a.masked_select(mask_1)
d = a.masked_select(mask_2)

torch.nonzero(input, out=None) → LongTensor 注意是返回的高维索引

>>> torch.nonzero(torch.Tensor([[0.6, 0.0, 0.0, 0.0],
...                             [0.0, 0.4, 0.0, 0.0],
...                             [0.0, 0.0, 1.2, 0.0],
...                             [0.0, 0.0, 0.0,-0.4]]))

 0  0
 1  1
 2  2
 3  3
[torch.LongTensor of size 4x2]

torch.squeeze(input, dim=None, out=None)/squeeze(dim=None) 超超超重要的函数
torch.stack(sequence, dim=0)
torch.t/t()
torch.transpose(input, dim0, dim1, out=None) → Tensor/transpose() 交换任意两个维度
torch.take(input, indices) → Tensor 展开之后的tensor取索引
tensor.permute(dims) 非常重要

x.permute(2, 0, 1)

torch.unbind(tensor, dim=0) 移除指定维度，返回一个truple，包含了沿着指定维切片后的各个切片
torch.unsqueeze(input, dim, out=None) 插入维度
torch.where(condition, x, y) → Tensor 注意condition (ByteTensor)

随机抽样 Random sampling

torch.manual_seed(seed)
torch.initial_seed() 注意做对比实验的时为了控制变量，多线程载入数据时每个线程的seed都需要严格设定
torch.get_rng_state() ->(ByteTensor)
torch.set_rng_state
torch.default_generator
torch.bernoulli(input, out=None) → Tensor伯努利投硬币，常用于样本的挖掘（hard example）
torch.multinomial(input, num_samples,replacement=False, out=None) → Longtensor 多项分布抽取样本
torch.normal(means, std, out=None) 离散正态分布中抽取随机数

torch.normal(means=torch.arange(1, 11), std=torch.arange(1, 0, -0.1))

 1.5104
 1.6955
 2.4895
 4.9185
 4.9895
 6.9155
 7.3683
 8.1836
 8.7164
 9.8916
[torch.FloatTensor of size 10]
>>> torch.normal(mean=0.5, std=torch.arange(1, 6))

  0.5723
  0.0871
 -0.3783
 -2.5689
 10.7893
[torch.FloatTensor of size 5]
>>> torch.normal(means=torch.arange(1, 6))

 1.1681
 2.8884
 3.7718
 2.5616
 4.2500
[torch.FloatTensor of size 5]

序列化 Serialization

torch.saves
torch.load

并行化 Parallelism

torch.get_num_threads
torch.set_num_threads(int)

数学操作Math operations

tensor有全部的对应数学函数
挑几个常用的：

ceil/floor/frac
round
torch.clamp(input, min, max, out=None) → Tensor 等价于tensorflow的tf.clip
torch.argmax(input, dim=None, keepdim=False)/torch.argmin(input, dim=None, keepdim=False)
torch.cumprod(input, dim, out=None) → Tensor $$y_i = x_1 \times x_2\times x_3\times \dots \times x_i$$/torch.prod(input, dim, keepdim=False, out=None) → Tensor
torch.cumsum(input, dim, out=None) → Tensor 同上
torch.dist(input, other, p=2) → Tensor p范数/torch.norm(input, p, dim, keepdim=False, out=None) → Tensor
- torch.mean(input, dim, keepdim=False, out=None) → Tensor 注意keep_dim是是否保持维度不变

>>> a = torch.randn(4, 4)
>>> a
tensor([[-0.3841,  0.6320,  0.4254, -0.7384],
        [-0.9644,  1.0131, -0.6549, -1.4279],
        [-0.2951, -1.3350, -0.7694,  0.5600],
        [ 1.0842, -0.9580,  0.3623,  0.2343]])
>>> torch.mean(a, 1)
tensor([-0.0163, -0.5085, -0.4599,  0.1807])
>>> torch.mean(a, 1, True)
tensor([[-0.0163],
        [-0.5085],
        [-0.4599],
        [ 0.1807]])

torch.median() 返回中间值

>>> a = torch.randn(1, 3)
>>> a
tensor([[ 1.5219, -1.5212,  0.2202]])
>>> torch.median(a)
tensor(0.2202)

torch.median(input, dim=-1, keepdim=False, values=None, indices=None) -> (Tensor, LongTensor)

>>> a = torch.randn(4, 5)
>>> a
tensor([[ 0.2505, -0.3982, -0.9948,  0.3518, -1.3131],
        [ 0.3180, -0.6993,  1.0436,  0.0438,  0.2270],
        [-0.2751,  0.7303,  0.2192,  0.3321,  0.2488],
        [ 1.0778, -1.9510,  0.7048,  0.4742, -0.7125]])
>>> torch.median(a, 1)
(tensor([-0.3982,  0.2270,  0.2488,  0.4742]), tensor([ 1,  4,  4,  3]))

torch.std(input, dim, keepdim=False, unbiased=True, out=None) → Tensor/torch.var(input, dim, keepdim=False, unbiased=True, out=None) → Tensor 标准差和方差
torch.sum(input, dim, keepdim=False, out=None) → Tensor
torch.unique(input, sorted=False, return_inverse=False) 元素去重（只能是1D tensor）

Comparison Ops

torch.eq(input, other, out=None) → Tensor other必须能广播，返回mask
torch.equal(tensor1, tensor2) → bool
torch.ge/gt/le/lt/ne/ ≥/＞/≤/＜
torch.topk(input, k, dim=None, largest=True, sorted=True, out=None) -> (Tensor, LongTensor)/torch.kthvalue(input, k, dim=None, keepdim=False, out=None) -> (Tensor, LongTensor)
torch.max(input) → Tensor
torch.max(input, dim, keepdim=False, out=None) -> (Tensor, LongTensor)
torch.max(input, other, out=None) → Tensor

>>> a = torch.randn(4)
>>> a
tensor([ 0.2942, -0.7416,  0.2653, -0.1584])
>>> b = torch.randn(4)
>>> b
tensor([ 0.8722, -1.7421, -0.4141, -0.5055])
>>> torch.max(a, b)
tensor([ 0.8722, -0.7416,  0.2653, -0.1584])

torch.min()也有三种方法，使用同max
torch.sort(input, dim=None, descending=False, out=None) -> (Tensor, LongTensor)

BLAS and LAPACK Operations

各种矩阵的基础运算

Spectral Ops

终于算是加上了。。

Tensor独有的Ops

tensor的前缀new_方法,是固定变量赋值，适用于ones，zeros，full，tensor（坑）

>>> tensor = torch.ones((2,), dtype=torch.int8)
>>> data = [[0, 1], [2, 3]]
>>> tensor.new_tensor(data)
tensor([[ 0,  1],[ 2,  3]], dtype=torch.int8)

torch.Tensor.item() 坑，注意只能是一个值，适合返回loss，acc等
apply_(callable) → Tensor(类似于map，python层面的cpu funtion，效率低)
cauchy_(median=0, sigma=1, *, generator=None) → Tensor
char(),byte(),double() ,int()
clone() /copy() 第一个是完全克隆，第二个是可广播的数值
contiguous() → Tensor 一些op为了高效运算，默认实现连续内存运算需求的，这时候要保证tensor的连续存储
is_contiguous() → bool/is_pinned()/is_cuda/is_pinned()/is_signed()
cpu()/cuda()
dim()
device
element_size() → int 返回变量类型的内存占用字节
expand(*sizes) → Tensor 重要：扩展dim维1的轴

>>> x = torch.tensor([[1], [2], [3]])
>>> x.size()
torch.Size([3, 1])
>>> x.expand(3, 4)
tensor([[ 1,  1,  1,  1],
        [ 2,  2,  2,  2],
        [ 3,  3,  3,  3]])
>>> x.expand(-1, 4)   # -1 means not changing the size of that dimension
tensor([[ 1,  1,  1,  1],
        [ 2,  2,  2,  2],
        [ 3,  3,  3,  3]])

index_copy_(dim, index, tensor) → Tensor 按索引复制元素

>>> x = torch.zeros(5, 3)
>>> t = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]], dtype=torch.float)
>>> index = torch.tensor([0, 4, 2])
>>> x.index_copy_(0, index, t)
tensor([[ 1.,  2.,  3.],
        [ 0.,  0.,  0.],
        [ 7.,  8.,  9.],
        [ 0.,  0.,  0.],
        [ 4.,  5.,  6.]])

index_fill_(dim, index, val) → Tensor
map_(tensor, callable)
Applies callable for each element in self tensor and the given tensor and stores the results in self tensor. self tensor and the given tensor must be broadcastable.
(待续)

一些坑

new_tensor 会新建变量，use torch.Tensor.requires_grad_() or torch.Tensor.detach()
mask_select index_select也会新建变量
reshape，resize，review
（待续）

posted @ 2018-07-04 01:11 yyhssg 阅读(5565) 评论(1) 编辑收藏举报

刷新页面返回顶部

yyhssg

pytorch文档阅读（一）

张量 Tensors

## 创建操作 Creation Ops

## 索引,切片,连接,编译操作

随机抽样 Random sampling

序列化 Serialization

并行化 Parallelism

数学操作Math operations

Comparison Ops

BLAS and LAPACK Operations

Spectral Ops

Tensor独有的Ops

一些坑

公告