(pytorch) I want to normalize [0 255] integer tensor to [0 1] float tensor

Question

I want to normalize [0 255] integer tensor to [0 1] float tensor.

I used cifar10 dataset and wanted to deal with integer image tensor.
so I made them integer tensor when I loaded dataset, I used "transforms.ToTensor()" so the values were set to [0 1] float

tensor([[[0.4588, 0.4588, 0.4588,  ..., 0.4980, 0.4980, 0.5020],
         [0.4706, 0.4706, 0.4706,  ..., 0.5098, 0.5098, 0.5137],
         [0.4824, 0.4824, 0.4824,  ..., 0.5216, 0.5216, 0.5294],
         ...,
         [0.3098, 0.3020, 0.2863,  ..., 0.4549, 0.3608, 0.3137],
         [0.2902, 0.2902, 0.2902,  ..., 0.4431, 0.3333, 0.3020],
         [0.2706, 0.2941, 0.2941,  ..., 0.4157, 0.3529, 0.3059]],

        [[0.7725, 0.7725, 0.7725,  ..., 0.7569, 0.7569, 0.7608],
         [0.7765, 0.7765, 0.7765,  ..., 0.7608, 0.7608, 0.7686],
         [0.7765, 0.7765, 0.7765,  ..., 0.7608, 0.7608, 0.7725],
         ...,
         [0.6510, 0.6314, 0.6078,  ..., 0.6941, 0.6510, 0.6392],
         [0.6314, 0.6235, 0.6118,  ..., 0.6784, 0.6196, 0.6275],
         [0.6157, 0.6235, 0.6157,  ..., 0.6549, 0.6431, 0.6314]],

To make them [0 255] integer tensor.

temp = np.floor(temp_images*256)
temp_int = torch.tensor(temp, dtype=torch.int32)
temp_images = torch.clamp(temp, 0, 255)

and the result was

torch.IntTensor
tensor([[[[ 94., 100., 100.,  ...,  98., 100., 102.],
          [ 86., 100., 101.,  ...,  83.,  91., 103.],
          [ 90., 100.,  99.,  ...,  80.,  66.,  86.],
          ...,
          [ 92.,  92.,  90.,  ...,  77., 107., 119.],
          [ 76.,  91., 100.,  ...,  95., 158., 170.],
          [ 86.,  83.,  87.,  ...,  97., 176., 205.]],

         [[105., 111., 111.,  ..., 109., 112., 113.],
          [ 97., 111., 112.,  ...,  94., 102., 114.],
          [101., 111., 110.,  ...,  90.,  77.,  97.],
          ...,
          [111., 110., 108.,  ...,  88., 120., 131.],
          [ 95., 108., 114.,  ..., 105., 165., 172.],
          [106., 100., 101.,  ..., 108., 183., 206.]],

         [[ 62.,  68.,  68.,  ...,  66.,  68.,  70.],
          [ 55.,  69.,  70.,  ...,  51.,  59.,  71.],
          [ 59.,  69.,  68.,  ...,  48.,  34.,  54.],
          ...,
          [ 59.,  59.,  56.,  ...,  54.,  95., 107.],
          [ 49.,  61.,  66.,  ...,  76., 152., 166.],
          [ 61.,  55.,  54.,  ...,  73., 170., 206.]]],

before forwarding them to the network, I want to make them [0 1] float tensor again.

So I tried

transforms.Normalize((0.5,0.5,0.5),(0.5,0.5,0.5))

But, the result is not normalized to [0 1] and rather it becomes bigger...!

tensor([[[117., 117., 117.,  ..., 127., 127., 128.],
         [120., 120., 120.,  ..., 130., 130., 131.],
         [123., 123., 123.,  ..., 133., 133., 135.],
         ...,
         [ 79.,  77.,  73.,  ..., 116.,  92.,  80.],
         [ 74.,  74.,  74.,  ..., 113.,  85.,  77.],
         [ 69.,  75.,  75.,  ..., 106.,  90.,  78.]],

        [[197., 197., 197.,  ..., 193., 193., 194.],
         [198., 198., 198.,  ..., 194., 194., 196.],
         [198., 198., 198.,  ..., 194., 194., 197.],

to

tensor([[[233., 233., 233.,  ..., 253., 253., 255.],
         [239., 239., 239.,  ..., 259., 259., 261.],
         [245., 245., 245.,  ..., 265., 265., 269.],
         ...,
         [157., 153., 145.,  ..., 231., 183., 159.],
         [147., 147., 147.,  ..., 225., 169., 153.],
         [137., 149., 149.,  ..., 211., 179., 155.]],

        [[393., 393., 393.,  ..., 385., 385., 387.],
         [395., 395., 395.,  ..., 387., 387., 391.],
         [395., 395., 395.,  ..., 387., 387., 393.],
         ...,
         [331., 321., 309.,  ..., 353., 331., 325.],
         [321., 317., 311.,  ..., 345., 315., 319.],
         [313., 317., 313.,  ..., 333., 327., 321.]],

How I can normalize [0 255] integer tensor to [0 1] float tensor?

Solution

The problem is that you seem to misunderstand what transforms.Normalize does. To quote from the PyTorch documentation:

Normalize a tensor image with mean and standard deviation. Given mean: (M1,...,Mn) and std: (S1,..,Sn) for n channels, this transform will normalize each channel of the input torch.*Tensor i.e. input[channel] = (input[channel] - mean[channel]) / std[channel]

The calculation for a value of, say 100, and the std and mean you provided would then be: 100 - 0.5 / 0.5 = 199. Of course, you could increase std and mean, but this does not guarantee you the exact result that you might expect. As suggested in the comments, the best way would probably be to invert the operations that you performed in order to get the tensor to [0 255] in the first place.

Edit:
As it turns out, according to this forum post, it seems that the transformations from PIL images to tensors automatically turn your value range to [0 1] (and to [0 255] if you transform to a PIL image, respectively), as is written in the fine-print of transforms.ToTensor. For the return transformation it is not explicitly stated, but can be enforced via the mode.

Comments

[ylee]

Thank you for your answer! I misunderstood it would be normalized to have mean 0.5 and std 0.5 if I use that function. I should have read function document more. Thank You for your answer.

[dennlinger]

No worries, this is what Stackoverflow is for ;-) In fact, I myself have not read enough in the documentation, since there seems to be an even better way to solve this! I'll update in a few minutes.