How to mock data as request.Response type in python

Question

I would like to write some testcase to exercise object_check in isinstance(obj, requests.Response) logic. After I create Mock data as return value for requests.post. The type for mock data is always be Mock class. In that way, how can I rewrite mock data so mock data can be type of requests.Response? so I can exercise line d = obj.json()?

from unittest.mock import patch, Mock
import unittest
import requests
from requests.exceptions import HTTPError
import pytest
def object_check(obj):
    if isinstance(obj, bytes):
        d = ujson.decode(obj.decode())
    elif isinstance(obj, requests.Response):
        d = obj.json()
    else:
        raise ValueError('invalid type')
    return d

def service_post(params):
    """
    trivial function that does a GET request
    against google, checks the status of the
    result and returns the raw content
    """
    url = "https://www.iamdomain.com"
    params = {'number': 1234, 'user_id': 1, 'name': 'john'}
    resp = requests.post(url, data=params)
    return object_check(resp)

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = Mock()
        r.status_code.return_value = 200
        r.json.return_value = data
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)

Solution 1

You could do this:

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = requests.Response()
        r.status_code = 200
        def json_func():
            return data
        r.json = json_func
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)

Test then passed for me when I ran it locally. Be aware that Mock is a mini-smell.

I used to be a big fan of Mock. As I've grown as a dev, though, I really try to avoid it. It can trick you into some really bad design, and they can be really hard to maintain (especially since you're modifying your Mock to hold return values). Mock can also create a false sense of security (your test will continue to pass even if the web services changes dramatically, so you might explode in prod). I don't think you really need it here. Two alternatives:

1. You could hit whatever service you're trying to hit, and serialize (save) that response out with pickle, and store to disk (save it in your test suite). Then have your unit test read it back in and use the actual response object. You'd still have to patch over requests.post, but at least the return values will be lined up for you and you won't have to add or modify them as your needs/application grows.
2. Just hit the web. Forget the patch entirely: just do the POST in your test and check the response. Of course, this might be slow, and will only work if you have internet. And you'll get goofy purists who will tell you to never to do this in a unit test. Maybe move it to an integration test if you run into one of those puristy people. But seriously, there's no substitute for doing what you're actually going to do in prod. The upside to doing this is that if the web service changes, then you'll know about it right away and can fix your code. Downside is it can slow down your test suite, and it's a potentially unreliable test (if the webservice is down, your test will fail...but it might actually be good to know that).

I recommend if the webservice is unstable (i.e liable to change), use option 2. Else, use option 1. Or do some combination of both (Mock and patch for a unit test, and hit the service on an integration test). Only you can decide!

HTH, good luck!

Solution 2

Use the spec argument when instantiating the mock:

>>> from unittest.mock import Mock
>>> from requests import Response
>>> m = Mock(spec=Response)
>>> m.__class__
requests.models.Response
>>> isinstance(m, Response)
True

Also note that r.status_code.return_value = 200 will not work with speccing; set the value directly instead:

r.status_code = 200

Solution 3

If you want to mock the text or content @property value use PropertyMock around the text

@patch.object(requests, 'post')
def test_service_post(mock_request_post):
    data = {'number': 0000, 'user_id': 0, 'name': 'john'}
    def res():
        r = requests.Response()
        r.status_code = 200
        type(r).text = mock.PropertyMock(return_value=my_text)  # property mock
        def json_func():
            return data
        r.json = json_func
        return r
    mock_request_post.return_value = res()
    assert data == service_post(data)