Python requests - POST data from a file

Question

I have used curl to send POST requests with data from files.

I am trying to achieve the same using python requests module. Here is my python script

import requests
payload=open('data','rb').read()
r = requests.post('https://IP_ADDRESS/rest/rest/2', auth=('userid', 'password'), data=payload , verify=False)
print r.text

Data file looks like below

'ID' : 'ISM03'

But my script is not POSTing the data from file. Am I missing something here.

In Curl , I used to have a command like below

Curl --data @filename -ik -X POST 'https://IP_ADDRESS/rest/rest/2' 

Solution

You do not need to use .read() here, simply stream the object directly. You do need to set the Content-Type header explicitly; curl does this when using --data but requests doesn't:

with open('data','rb') as payload:
    headers = {'content-type': 'application/x-www-form-urlencoded'}
    r = requests.post('https://IP_ADDRESS/rest/rest/2', auth=('userid', 'password'),
                      data=payload, verify=False, headers=headers)

I've used the open file object as a context manager so that it is also auto-closed for you when the block exits (e.g. an exception occurs or requests.post() successfully returns).

Comments

[Games Brainiac]

How is it, that you always seem to answer even the hardest python questions?

[Martijn Pieters]

Don't believe everything I post.. There was an error in the first revision there.

[skanagasabap]

@MartijnPieters thanks for the reply.. I still couldn't POST the data from the file. If I use the values directly in the script it works. ex- payload={ 'ID' : 'ISM03' }. But when I have the contents in a file and use as you have mentioned, I couldn't update. Do I need to change the format in the input file ?

[Martijn Pieters]

curl expects the data file to be properly encoded already, and so does requests. I assumed you wanted it posted as is, just like curl --data @filename would do.

[Martijn Pieters]

When you use payload = {'ID' : 'ISM03'} you are creating a python dictionary instead, and that then is encoded in the right format (ID=ISM03 in this specific case). If you put that into the data file it'll work in both curl and requests.

[skanagasabap]

@MartijnPieters Great piece of information... thanks . Able to achieve it now.

[wobmene]

does requests close the file for me after posting or should I close it myself?

[Martijn Pieters]

@khrf: It doesn't close it, no. Updated to use the file object as a context manager to ensure it is closed.