'DataLoader' object does not support indexing
Solution 1
Solution
input_transform = standard_transforms.Compose([
transforms.Resize((255,255)), # to Make sure all the
transforms.CenterCrop(224), # imgs are at the same size
transforms.ToTensor()
])
# torch.utils.data.Dataset object
trainset = datasets.ImageNet('/media/farshid/DataStore/temp/Imagenet/',
split='train', download=False, transform = input_transform)
# torch.utils.data.DataLoader object
trainloader =torch.utils.data.DataLoader(trainset, batch_size=2, shuffle=False)
for batch_idx, data in enumerate(trainloader, 0):
x, y = data
break
Solution 2
Well, the answer is pretty simple (besides error mentioned in the other answer).
DataLoader
has no __getitem__
method (see in the source code for yourself).
It is used for iterating, not random access, over data (or batches of data). If you want to access specific element you should use torch.utils.data.Dataset
, in your case:
trainset = torchvision.datasets.ImageNet('/media/farshid/DataStore/temp/Imagenet/', split='train', )
trainset[0]
Getting a batch
If you want to get a batch you may iterate over it and break afterwards:
for batch in dataloader:
print(batch) # or anything else you want to do
break
DataLoader
creates random indices in default or specified way (see samplers), hence there is no __getitem__
as it wouldn't make sense for this object.
You may also inherit from the DataLoader
and create your own __getitem__
function doing what you want (more complicated though).
Full example
# torch.utils.data.Dataset object
trainset = datasets.ImageNet('/media/farshid/DataStore/temp/Imagenet/', split='train', download=True)
# torch.utils.data.DataLoader object
trainloader =torch.utils.data.DataLoader(trainset, batch_size=1, shuffle=False)
for batch in trainloader:
print(batch)
break
Above should print the first batch whatever is inside.
Solution 3
The input dataset to torch.utils.data.DataLoader()
should be of type torch.utils.data.Dataset
, not torch.utils.data.DataLoader
, which is what you are doing in above code.
So, your above code should be:
trainset = torchvision.datasets.ImageNet('/media/farshid/DataStore/temp/Imagenet/',
split='train',
download=False)
trainloader = torch.utils.data.DataLoader(trainset,
batch_size=1,
shuffle=False,
num_workers=1)
For more details, check the official torch documentation here.
Farshid Rayhan
Hit me up if you want to collaborate on a computer vision project I will be more than happy to provide contribution.
Updated on June 05, 2022Comments
-
Farshid Rayhan almost 2 years
I have downloaded the ImageNet dataset via this pytorch api by setting download=True. But I cannot iterate through the dataloader.
The error says "'DataLoader' object does not support indexing"
trainset = torch.utils.data.DataLoader( datasets.ImageNet('/media/farshid/DataStore/temp/Imagenet/', split='train', download=False)) trainloader = torch.utils.data.DataLoader(trainset, batch_size=1, shuffle=False, num_workers=1)
I tried a simple approach I just tried to run the following,
trainloader[0]
In the root directory, the pattern is
root/ train/ n01440764/ n01443537/ n01443537_2.jpg
The docs in the official website doesnt say anything else. https://pytorch.org/docs/stable/torchvision/datasets.html#imagenet
What am I doing wrong ?
-
Farshid Rayhan almost 5 yearsYes I see the problem and I tried your solution. I still have the same error "'DataLoader' object does not support indexing" when I do "trainloader[0]"
-
Szymon Maszke almost 5 yearsWhile true, it does not solve the issue (leaving alone the fact of reiterating the comment).
-
Farshid Rayhan almost 5 yearsThen how do I get it like a batch ?
-
Farshid Rayhan almost 5 yearsJust to clarify, what do you mean by 'dataloader' according to Anubhav Singh's code is it the 'trainset' or 'trainloader' ? Cause with trainloader it doesnt work !
-
Szymon Maszke almost 5 yearsIt works if you create
DataLoader
fromtrain_dataset
.dataloader
refers to instance of suchDataLoader
class. -
Farshid Rayhan almost 5 yearsI am really getting confused here. Can you please explain or just update your codes
-
Farshid Rayhan almost 5 yearsWell yes and no. The problem I was facing was the img being a PIL file and the debugger doesnt say it. Your code still doesnt run without errors. The fix is add "standard_transforms.ToTensor()," as transforms. :)
-
Farshid Rayhan almost 5 yearsPlus for some reason increasing the batch size from 1 to anything else doesnt work
-
Szymon Maszke almost 5 yearsYeah, I assumed correct transformations and
print
is a simple example to get the point across.