How do i download pdf file over https with python

Question

I am writing a python script, which will save pdf file locally according to the format given in URL. for eg.

https://Hostname/saveReport/file_name.pdf   #saves the content in PDF file.

I am opening this URL through python script :

 import webbrowser
 webbrowser.open("https://Hostname/saveReport/file_name.pdf")  

The url contains lots of images and text. Once this URL is opened i want to save a file in pdf format using python script.

This is what i have done so far.
Code 1:

import requests
url="https://Hostname/saveReport/file_name.pdf"    #Note: It's https
r = requests.get(url, auth=('usrname', 'password'), verify=False)
file = open("file_name.pdf", 'w')
file.write(r.read())
file.close()

Code 2:

 import urllib2
 import ssl
 url="https://Hostname/saveReport/file_name.pdf"
 context = ssl._create_unverified_context()
 response = urllib2.urlopen(url, context=context)  #How should i pass authorization details here?
 html = response.read()

In above code i am getting: urllib2.HTTPError: HTTP Error 401: Unauthorized

If i use Code 2, how can i pass authorization details?

Solution 1

I think this will work

import requests
import shutil
url="https://Hostname/saveReport/file_name.pdf"    #Note: It's https
r = requests.get(url, auth=('usrname', 'password'), verify=False,stream=True)
r.raw.decode_content = True
with open("file_name.pdf", 'wb') as f:
        shutil.copyfileobj(r.raw, f)

Solution 2

One way you can do that is:

import urllib3
urllib3.disable_warnings()
url = r"https://websitewithfile.com/file.pdf"
fileName = r"file.pdf"
with urllib3.PoolManager() as http:
    r = http.request('GET', url)
    with open(fileName, 'wb') as fout:
        fout.write(r.data)

Comments

[matth]

response.text is probably a bad choice, since it involves a decoding step. Maybe open the file with wb and write response.content instead.

[Joran Beasley]

response.raw I think

[matth]

Apparently, response.iter_chunk is request's "preferred and recommended way to retrieve a document": docs.python-requests.org/en/latest/user/quickstart/…

[Raphaël Vigée]

I edited with a better way of getting response content