Django Rest Framework File Upload

Question

I am using Django Rest Framework and AngularJs to upload a file. My view file looks like this:

class ProductList(APIView):
    authentication_classes = (authentication.TokenAuthentication,)
    def get(self,request):
        if request.user.is_authenticated(): 
            userCompanyId = request.user.get_profile().companyId
            products = Product.objects.filter(company = userCompanyId)
            serializer = ProductSerializer(products,many=True)
            return Response(serializer.data)

    def post(self,request):
        serializer = ProductSerializer(data=request.DATA, files=request.FILES)
        if serializer.is_valid():
            serializer.save()
            return Response(data=request.DATA)

As the last line of post method should return all the data, I have several questions:

• how to check if there is anything in request.FILES?
• how to serialize file field?
• how should I use parser?

Solution 1

Use the FileUploadParser, it's all in the request. Use a put method instead, you'll find an example in the docs :)

class FileUploadView(views.APIView):
    parser_classes = (FileUploadParser,)

    def put(self, request, filename, format=None):
        file_obj = request.FILES['file']
        # do some stuff with uploaded file
        return Response(status=204)

Solution 2

Editor's note:

• This answer uses pre_save, which no longer exists in Django REST framework 3.0.
• In a sufficiently new version of Django REST framework, MultiPartParser should be available by default, which allows uploading file with no special handling. See an answer below for an example.

I'm using the same stack and was also looking for an example of file upload, but my case is simpler since I use the ModelViewSet instead of APIView. The key turned out to be the pre_save hook. I ended up using it together with the angular-file-upload module like so:

# Django
class ExperimentViewSet(ModelViewSet):
    queryset = Experiment.objects.all()
    serializer_class = ExperimentSerializer

    def pre_save(self, obj):
        obj.samplesheet = self.request.FILES.get('file')

class Experiment(Model):
    notes = TextField(blank=True)
    samplesheet = FileField(blank=True, default='')
    user = ForeignKey(User, related_name='experiments')

class ExperimentSerializer(ModelSerializer):
    class Meta:
        model = Experiment
        fields = ('id', 'notes', 'samplesheet', 'user')

// AngularJS
controller('UploadExperimentCtrl', function($scope, $upload) {
    $scope.submit = function(files, exp) {
        $upload.upload({
            url: '/api/experiments/' + exp.id + '/',
            method: 'PUT',
            data: {user: exp.user.id},
            file: files[0]
        });
    };
});

Solution 3

Finally I am able to upload image using Django. Here is my working code

views.py

class FileUploadView(APIView):
    parser_classes = (FileUploadParser, )

    def post(self, request, format='jpg'):
        up_file = request.FILES['file']
        destination = open('/Users/Username/' + up_file.name, 'wb+')
        for chunk in up_file.chunks():
            destination.write(chunk)
        destination.close()  # File should be closed only after all chuns are added

        # ...
        # do some stuff with uploaded file
        # ...
        return Response(up_file.name, status.HTTP_201_CREATED)

urls.py

urlpatterns = patterns('', 
url(r'^imageUpload', views.FileUploadView.as_view())

curl request to upload

curl -X POST -S -H -u "admin:password" -F "[email protected];type=image/jpg" 127.0.0.1:8000/resourceurl/imageUpload

Solution 4

From my experience, you don't need to do anything particular about file fields, you just tell it to make use of the file field:

from rest_framework import routers, serializers, viewsets

class Photo(django.db.models.Model):
    file = django.db.models.ImageField()

    def __str__(self):
        return self.file.name

class PhotoSerializer(serializers.ModelSerializer):
    class Meta:
        model = models.Photo
        fields = ('id', 'file')   # <-- HERE

class PhotoViewSet(viewsets.ModelViewSet):
    queryset = models.Photo.objects.all()
    serializer_class = PhotoSerializer

router = routers.DefaultRouter()
router.register(r'photos', PhotoViewSet)

api_urlpatterns = ([
    url('', include(router.urls)),
], 'api')
urlpatterns += [
    url(r'^api/', include(api_urlpatterns)),
]

and you're ready to upload files:

curl -sS http://example.com/api/photos/ -F 'file=@/path/to/file'

Add -F field=value for each extra field your model has. And don't forget to add authentication.

Solution 5

After spending 1 day on this, I figured out that ...

For someone who needs to upload a file and send some data, there is no straight fwd way you can get it to work. There is an open issue in json api specs for this. One possibility i have seen is to use multipart/related as shown here, but i think its very hard to implement it in drf.

Finally what i had implemented was to send the request as formdata. You would send each file as file and all other data as text. Now for sending the data as text you have two choices. case 1) you can send each data as key value pair or case 2) you can have a single key called data and send the whole json as string in value.

The first method would work out of the box if you have simple fields, but will be a issue if you have nested serializes. The multipart parser wont be able to parse the nested fields.

Below i am providing the implementation for both the cases

Models.py

class Posts(models.Model):
    id = models.UUIDField(default=uuid.uuid4, primary_key=True, editable=False)
    caption = models.TextField(max_length=1000)
    media = models.ImageField(blank=True, default="", upload_to="posts/")
    tags = models.ManyToManyField('Tags', related_name='posts')

serializers.py -> no special changes needed, not showing my serializer here as its too lengthy because of the writable ManyToMany Field implimentation.

views.py

class PostsViewset(viewsets.ModelViewSet):
    serializer_class = PostsSerializer
    #parser_classes = (MultipartJsonParser, parsers.JSONParser) use this if you have simple key value pair as data with no nested serializers
    #parser_classes = (parsers.MultipartParser, parsers.JSONParser) use this if you want to parse json in the key value pair data sent
    queryset = Posts.objects.all()
    lookup_field = 'id'

Now, if you are following the first method and is only sending non-Json data as key value pairs, you don't need a custom parser class. DRF'd MultipartParser will do the job. But for the second case or if you have nested serializers (like i have shown) you will need custom parser as shown below.

utils.py

from django.http import QueryDict
import json
from rest_framework import parsers

class MultipartJsonParser(parsers.MultiPartParser):

    def parse(self, stream, media_type=None, parser_context=None):
        result = super().parse(
            stream,
            media_type=media_type,
            parser_context=parser_context
        )
        data = {}

        # for case1 with nested serializers
        # parse each field with json
        for key, value in result.data.items():
            if type(value) != str:
                data[key] = value
                continue
            if '{' in value or "[" in value:
                try:
                    data[key] = json.loads(value)
                except ValueError:
                    data[key] = value
            else:
                data[key] = value

        # for case 2
        # find the data field and parse it
        data = json.loads(result.data["data"])

        qdict = QueryDict('', mutable=True)
        qdict.update(data)
        return parsers.DataAndFiles(qdict, result.files)

This serializer would basically parse any json content in the values.

The request example in post man for both cases: case 1 ,

Case 2

View more solutions

Comments

[psychok7]

hey,do you know how i could solve stackoverflow.com/questions/26673572/… ?

[makerj]

why destination.close() is placed at inside of for loop?

[Chuck Wilbur]

Seems it would be better to use with open('/Users/Username/' + up_file.name, 'wb+') as destination: and remove the close entirely

[Md. Tanvir Raihan]

@pleasedontbelong why PUT method has been used here instead of POST?

[chrizonline]

hi @pleasedontbelong, if it's creating a new record, would it be POST instead? and will it still work with FileUploadParser?

[Guy S]

pre_save is deprecated in drf 3.x

[dudeman]

@pleasedontbelong RTan asks a pretty good question. Reading RFC-2616 provides a subtlety I wasn't aware of until now. "The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request"

[liquidki]

On the current version of DRF 3.8.2, it will parse by default application/json, application/x-www-form-urlencoded, and multipart/form-data.

[x-yuri]

Why FileUploadParser? "The FileUploadParser is for usage with native clients that can upload the file as a raw data request. For web-based uploads, or for native clients with multipart upload support, you should use the MultiPartParser parser instead." Doesn't seem like a good option generally. What's more, I don't see file uploads needing any particular treatment.

[x-yuri]

So you suggest uploading a file, then attaching it to some db record. What if attaching never happens for some reason? Why not do it in one request? parser_classes is not there to limit which files can be uploaded. It let's you decide which formats can be used to make requests. On second thought, the way you handle the upload... it seems like you're putting data from CSV into database. Not what OP asked.

[x-yuri]

What validation logic does FileUploaderSerializer.validate method contain?

[x-yuri]

From my experience, no special treatment is needed for file fields.

[x-yuri]

It's simpler to use ModelViewSet. Also, they most likely implemented it better.

[x-yuri]

I'd rather avoid case 2. Creating one database record per request should be fine most of the time.

[Wolfgang Leon]

@x-yuri by saying "a CSV is a file" and the question is; How to check if there's data in the request? By using this method, you'll find the data in request.data. _data = request.data due PUT is being used. Like you said, parser_classes are there to decide which formats CAN be used to make request for hence by using any other format that you DON'T want, will then be excluded adding an extra layer of security. What yo do with your data is up to you. Using "Try Except" you can check if "attaching never happens" tho there's no need for it, that's not what the code does. These is made in 1 request

[David Zwart]

To second @x-yuri, DRF complains about the Content-Disposition header being empty when I use the FileUploadParser. MultiPartParser is much simpler, since it just assumes the filename to be the given filename in the Form fields.

[Rufat]

@Guy-S, perform_create, perform_update, perform_destroy methods replace the old-style version 2.x pre_save, post_save, pre_delete and post_delete methods, which are no longer available: django-rest-framework.org/api-guide/generic-views/#methods

[Metehan Gülaç]

very helpful thank you a lot. But i don't understand, why you are converting dict data to QueryDict in parser? In my case in Django, normal dictionary data work perfectly without converting.

[Metehan Gülaç]

I tried a different scenario using the answer you mentioned and it works successfully. you can look at my answer.

[Olivier Pons]

I've been relying on this answerr for the whole day... until I found that when you want to upload multiple files, it's not FileUploadParser that is needed, but MultiPartParser!

[Eduard Grigoryev]

And how could we send the request using ajax. What is imageUrl actually?

[sadat]

the imageUrl is the file in the request.

[sadat]

if this works stackoverflow.com/questions/64547729/… should work too., but it does not.

[DJ Ramones]

This. I'm not sure what the fuss is all about in all the higher-voted answers, maybe fiddling with parsers and all was necessary with earlier versions of DRF, but as of 2021 the generic classes just work for the basic use case, which is what the questions seems to be mainly about.

[Pluviophile]

COuld you add an explanation to your code?

[nicbou]

This answer could use a bit of context

[hassan ketabi]

if you change the file in your upload function to samplesheet you can ignore pre_save like in stackoverflow.com/a/69544111/6200607