Python standard library to POST multipart/form-data encoded data

Question

I would like to POST multipart/form-data encoded data. I have found an external module that does it: http://atlee.ca/software/poster/index.html however I would rather avoid this dependency. Is there a way to do this using the standard libraries?

thanks

Solution 1

The standard library does not currently support that. There is cookbook recipe that includes a fairly short piece of code that you just may want to copy, though, along with long discussions of alternatives.

Solution 2

It's an old thread but still a popular one, so here is my contribution using only standard modules.

The idea is the same than here but support Python 2.x and Python 3.x. It also has a body generator to avoid unnecessarily memory usage.

import codecs
import mimetypes
import sys
import uuid
try:
    import io
except ImportError:
    pass # io is requiered in python3 but not available in python2

class MultipartFormdataEncoder(object):
    def __init__(self):
        self.boundary = uuid.uuid4().hex
        self.content_type = 'multipart/form-data; boundary={}'.format(self.boundary)

    @classmethod
    def u(cls, s):
        if sys.hexversion < 0x03000000 and isinstance(s, str):
            s = s.decode('utf-8')
        if sys.hexversion >= 0x03000000 and isinstance(s, bytes):
            s = s.decode('utf-8')
        return s

    def iter(self, fields, files):
        """
        fields is a sequence of (name, value) elements for regular form fields.
        files is a sequence of (name, filename, file-type) elements for data to be uploaded as files
        Yield body's chunk as bytes
        """
        encoder = codecs.getencoder('utf-8')
        for (key, value) in fields:
            key = self.u(key)
            yield encoder('--{}\r\n'.format(self.boundary))
            yield encoder(self.u('Content-Disposition: form-data; name="{}"\r\n').format(key))
            yield encoder('\r\n')
            if isinstance(value, int) or isinstance(value, float):
                value = str(value)
            yield encoder(self.u(value))
            yield encoder('\r\n')
        for (key, filename, fd) in files:
            key = self.u(key)
            filename = self.u(filename)
            yield encoder('--{}\r\n'.format(self.boundary))
            yield encoder(self.u('Content-Disposition: form-data; name="{}"; filename="{}"\r\n').format(key, filename))
            yield encoder('Content-Type: {}\r\n'.format(mimetypes.guess_type(filename)[0] or 'application/octet-stream'))
            yield encoder('\r\n')
            with fd:
                buff = fd.read()
                yield (buff, len(buff))
            yield encoder('\r\n')
        yield encoder('--{}--\r\n'.format(self.boundary))

    def encode(self, fields, files):
        body = io.BytesIO()
        for chunk, chunk_len in self.iter(fields, files):
            body.write(chunk)
        return self.content_type, body.getvalue()

Demo

# some utf8 key/value pairs
fields = [('প্রায়', 42), ('bar', b'23'), ('foo', 'ން:')]
files = [('myfile', 'image.jpg', open('image.jpg', 'rb'))]

# iterate and write chunk in a socket
content_type, body = MultipartFormdataEncoder().encode(fields, files)

Solution 3

You can't do this with the stdlib quickly. Howevewr, see the MultiPartForm class in this PyMOTW. You can probably use or modify that to accomplish whatever you need:

• PyMOTW: urllib2 - Library for opening URLs.

Comments

[Michael Herrmann]

Finally a solution that works for Python 3 with the standard library only.

[ekhumoro]

PyMOTW-3: urllib.request - Uploading Files.