PyTorch: Expected input batch_size (12) to match target batch_size (64)
The error occurs because your model output, out
, has shape (12, 10)
, while your target
has a length of 64.
Since you are using a batch size of 64 and predicting the probabilities of 10 classes, you would expect your model output to be of shape (64, 10)
, so clearly there is something amiss in the forward()
method.
Going through it line by line and noting the size of x
at every step, we can try to find out what is going wrong:
...
# x.shape = (64, 20, 4, 4) at this point as seen in your print statement
x = x.view(-1, 320) # x.shape = (64, 320)
x = self.fc1(x) # x.shape = (64, 60)
x = x.view(-1, 320) # x.shape = (12, 320)
x = F.relu(self.fc1(x)) # x.shape = (12, 60)
x = self.fc2(x) # x.shape = (12, 10)
return F.log_softmax(x, dim=0) # x.shape = (12, 10)
What you actually most likely want is:
...
# x.shape = (64, 20, 4, 4) at this point as seen in your print statement
x = x.view(-1, 320) # x.shape = (64, 320)
x = F.relu(self.fc1(x)) # x.shape = (64, 60)
x = self.fc2(x) # x.shape = (64, 10)
return F.log_softmax(x, dim=1) # x.shape = (64, 10)
Note: While not related to the error, note also that you want to softmax over dim=1
since that is the dimension that contains the logits for the classes.
Related videos on Youtube
![Smighty](https://i.stack.imgur.com/1F9Zj.jpg?s=256&g=1)
Smighty
Updated on July 02, 2022Comments
-
Smighty almost 2 years
I tried out PyTorch and wanted to write a program for MNIST. But, I got the error message:
Expected input batch_size (12) to match target batch_size (64)
I searched for a solution but I don't understand what's wrong with my code.
#kwargs is empty because I don't use cuda kwargs = {} train_data = torch.utils.data.DataLoader( datasets.MNIST('data', train=True, download=True, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,),(0.3081,))])), batch_size=64, shuffle=True, **kwargs) test_data = torch.utils.data.DataLoader( datasets.MNIST('data', train=False, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,),(0.3081,))])), batch_size=64, shuffle=True, **kwargs) class Netz(nn.Module): def __init__(self): super(Netz, self).__init__() self.conv1 = nn.Conv2d(1,10, kernel_size=5) self.conv2 = nn.Conv2d(10, 20, kernel_size=5) self.conv_dropout = nn.Dropout2d() self.fc1 = nn.Linear(320, 60) self.fc2 = nn.Linear(60, 10) def forward(self, x): x = self.conv1(x) x = F.max_pool2d(x, 2) x = F.relu(x) x = self.conv2(x) x = self.conv_dropout(x) x = F.max_pool2d(x, 2) x = F.relu(x) print(x.shape) x = x.view(-1, 320) x = self.fc1(x) x = x.view(-1, 320) x = F.relu(self.fc1(x)) x = self.fc2(x) return F.log_softmax(x, dim=0) model = Netz() optimizer = optim.SGD(model.parameters(), lr=0.1, momentum=0.8) def train(epoch): model.train() for batch_id, (data, target) in enumerate(train_data): data = Variable(data) target = Variable(target) optimizer.zero_grad() out = model(data) print(out.shape) criterion = nn.CrossEntropyLoss() loss = criterion(out, target) loss.backward() optimizer.step() print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'. format( epoch, batch_id * len(data), len(train_data.dataset), 100. * batch_id / len(train_data), loss.data[0]))
The output should show the epoch and some other information. Actually, I print out the shape of my tensor but I don't know what's wrong. Here is the error message:
/home/michael/Programmierung/Python/PyTorch/venv/bin/python /home/michael/Programmierung/Python/PyTorch/mnist.py torch.Size([64, 20, 4, 4]) torch.Size([12, 10]) Traceback (most recent call last): File "/home/michael/Programmierung/Python/PyTorch/mnist.py", line 69, in <module> train(epoch) File "/home/michael/Programmierung/Python/PyTorch/mnist.py", line 60, in train loss = criterion(out, target) File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__ result = self.forward(*input, **kwargs) File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 942, in forward ignore_index=self.ignore_index, reduction=self.reduction) File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 2056, in cross_entropy return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction) File "/home/michael/Programmierung/Python/PyTorch/venv/lib/python3.6/site-packages/torch/nn/functional.py", line 1869, in nll_loss .format(input.size(0), target.size(0))) ValueError: Expected input batch_size (12) to match target batch_size (64). Process finished with exit code 1