Difference between tensor.permute and tensor.view in PyTorch?

Question

What is the difference between tensor.permute() and tensor.view()?

They seem to do the same thing.

Solution 1

---

Input

In [12]: aten = torch.tensor([[1, 2, 3], [4, 5, 6]])

In [13]: aten
Out[13]: 
tensor([[ 1,  2,  3],
        [ 4,  5,  6]])

In [14]: aten.shape
Out[14]: torch.Size([2, 3])

---

torch.view() reshapes the tensor to a different but compatible shape. For example, our input tensor aten has the shape (2, 3). This can be viewed as tensors of shapes (6, 1), (1, 6) etc.,

# reshaping (or viewing) 2x3 matrix as a column vector of shape 6x1
In [15]: aten.view(6, -1)
Out[15]: 
tensor([[ 1],
        [ 2],
        [ 3],
        [ 4],
        [ 5],
        [ 6]])

In [16]: aten.view(6, -1).shape
Out[16]: torch.Size([6, 1])

Alternatively, it can also be reshaped or viewed as a row vector of shape (1, 6) as in:

In [19]: aten.view(-1, 6)
Out[19]: tensor([[ 1,  2,  3,  4,  5,  6]])

In [20]: aten.view(-1, 6).shape
Out[20]: torch.Size([1, 6])

---

Whereas tensor.permute() is only used to swap the axes. The below example will make things clear:

In [39]: aten
Out[39]: 
tensor([[ 1,  2,  3],
        [ 4,  5,  6]])

In [40]: aten.shape
Out[40]: torch.Size([2, 3])

# swapping the axes/dimensions 0 and 1
In [41]: aten.permute(1, 0)
Out[41]: 
tensor([[ 1,  4],
        [ 2,  5],
        [ 3,  6]])

# since we permute the axes/dims, the shape changed from (2, 3) => (3, 2)
In [42]: aten.permute(1, 0).shape
Out[42]: torch.Size([3, 2])

You can also use negative indexing to do the same thing as in:

In [45]: aten.permute(-1, 0)
Out[45]: 
tensor([[ 1,  4],
        [ 2,  5],
        [ 3,  6]])

In [46]: aten.permute(-1, 0).shape
Out[46]: torch.Size([3, 2])

Solution 2

View changes how the tensor is represented. For ex: a tensor with 4 elements can be represented as 4X1 or 2X2 or 1X4 but permute changes the axes. While permuting the data is moved but with view data is not moved but just reinterpreted.

Below code examples may help you. a is 2x2 tensor/matrix. With the use of view you can read a as a column or row vector (tensor). But you can't transpose it. To transpose you need permute. Transpose is achieved by swapping/permuting axes.

In [7]: import torch

In [8]: a = torch.tensor([[1,2],[3,4]])

In [9]: a
Out[9]: 
tensor([[ 1,  2],
        [ 3,  4]])

In [11]: a.permute(1,0)
Out[11]: 
tensor([[ 1,  3],
        [ 2,  4]])

In [12]: a.view(4,1)
Out[12]: 
tensor([[ 1],
        [ 2],
        [ 3],
        [ 4]])

In [13]: 

Bonus: See https://twitter.com/karpathy/status/1013322763790999552

Solution 3

tensor.permute() permutes the order of the axes of a tensor.

tensor.view() reshapes the tensor (analogous to numpy.reshape) by reducing/expanding the size of each dimension (if one increases, the others must decrease).