Nested validation with the flask-restful RequestParser

Question

Using the flask-restful micro-framework, I am having trouble constructing a RequestParser that will validate nested resources. Assuming an expected JSON resource format of the form:

{
    'a_list': [
        {
            'obj1': 1,
            'obj2': 2,
            'obj3': 3
        },
        {
            'obj1': 1,
            'obj2': 2,
            'obj3': 3
        }
    ]
}

Each item in a_list corresponds to an object:

class MyObject(object):
    def __init__(self, obj1, obj2, obj3)
        self.obj1 = obj1
        self.obj2 = obj2
        self.obj3 = obj3

... and one would then create a RequestParser using a form something like:

from flask.ext.restful import reqparse
parser = reqparse.RequestParser()
parser.add_argument('a_list', type=MyObject, action='append')

... but how would you validate the nested MyObjects of each dictionary inside a_list? Or, alternately, is this the wrong approach?

The API this corresponds to treats each MyObject as, essentially, an object literal, and there may be one or more of them passed to the service; therefore, flattening the resource format will not work for this circumstance.

Solution 1

I have had success by creating RequestParser instances for the nested objects. Parse the root object first as you normally would, then use the results to feed into the parsers for the nested objects.

The trick is the location argument of the add_argument method and the req argument of the parse_args method. They let you manipulate what the RequestParser looks at.

Here's an example:

root_parser = reqparse.RequestParser()
root_parser.add_argument('id', type=int)
root_parser.add_argument('name', type=str)
root_parser.add_argument('nested_one', type=dict)
root_parser.add_argument('nested_two', type=dict)
root_args = root_parser.parse_args()

nested_one_parser = reqparse.RequestParser()
nested_one_parser.add_argument('id', type=int, location=('nested_one',))
nested_one_args = nested_one_parser.parse_args(req=root_args)

nested_two_parser = reqparse.RequestParser()
nested_two_parser.add_argument('id', type=int, location=('nested_two',))
nested_two_args = nested_two_parser.parse_args(req=root_args)

Solution 2

I would suggest using a data validation tool such as cerberus. You start by defining a validation schema for your object (Nested object schema is covered in this paragraph), then use a validator to validate the resource against the schema. You also get detailed error messages when the validation fails.

In the following example, I want to validate a list of locations:

from cerberus import Validator
import json


def location_validator(value):
    LOCATION_SCHEMA = {
        'lat': {'required': True, 'type': 'float'},
        'lng': {'required': True, 'type': 'float'}
    }
    v = Validator(LOCATION_SCHEMA)
    if v.validate(value):
        return value
    else:
        raise ValueError(json.dumps(v.errors))

The argument is defined as follows:

parser.add_argument('location', type=location_validator, action='append')

Solution 3

Since the type argument here is nothing but a callable that either returns a parsed value or raise ValueError on invalid type, I would suggest creating your own type validator for this. The validator could look something like:

from flask.ext.restful import reqparse
def myobj(value):
    try:
        x = MyObj(**value)
    except TypeError:
        # Raise a ValueError, and maybe give it a good error string
        raise ValueError("Invalid object")
    except:
        # Just in case you get more errors
        raise ValueError 

    return x


#and now inside your views...
parser = reqparse.RequestParser()
parser.add_argument('a_list', type=myobj, action='append')

Solution 4

I found the bbenne10s answer really useful, but it didn't work for me as is.

The way I did this is probably wrong, but it works. My problem is that I don't understand what action='append' does as what it seems to do is wrap the value received in a list, but it doesn't make any sense to me. Can someone please explain whats the point of this in the comments?

So what I ended up doing is creating my own listtype, get the list inside the value param and then iterate through the list this way:

from flask.ext.restful import reqparse
def myobjlist(value):
    result = []
    try:
        for v in value:
            x = MyObj(**v)
            result.append(x)
    except TypeError:
        raise ValueError("Invalid object")
    except:
        raise ValueError

    return result


#and now inside views...
parser = reqparse.RequestParser()
parser.add_argument('a_list', type=myobjlist)

Not a really elegant solution, but at least it does the work. I hope some one can point us in the right direction...

Update

As bbenne10 has said in the comments, what action='append' does is append all the arguments named the same into a list, so in the case of the OP, it doesn't seem to be very useful.

I have iterated over my solution because I didn't like the fact that reqparse wasn't parsing/validating any of the nested objects so I what I have done is use reqparse inside the custom object type myobjlist.

First, I have declared a new subclass of Request, to pass it as the request when parsing the nested objects:

class NestedRequest(Request):
    def __init__(self, json=None, req=request):
        super(NestedRequest, self).__init__(req.environ, False, req.shallow)
        self.nested_json = json

    @property
    def json(self):
        return self.nested_json

This class overrides the request.json so that it uses a new json with the object to being parsed. Then, I added a reqparse parser to myobjlist to parse all the arguments and added an except to catch the parsing error and pass the reqparse message.

from flask.ext.restful import reqparse
from werkzeug.exceptions import ClientDisconnected
def myobjlist(value):
    parser = reqparse.RequestParser()
    parser.add_argument('obj1', type=int, required=True, help='No obj1 provided', location='json')
    parser.add_argument('obj2', type=int, location='json')
    parser.add_argument('obj3', type=int, location='json')
    nested_request = NestedRequest()
    result = []
    try:
        for v in value:
            nested_request.nested_json = v
            v = parser.parse_args(nested_request)
            x = MyObj(**v)
            result.append(x)
    except TypeError:
        raise ValueError("Invalid object")
    except ClientDisconnected, e:
        raise ValueError(e.data.get('message', "Parsing error") if e.data else "Parsing error")
    except:
        raise ValueError
    return result

This way, even the nested objects will get parsed through reqparse and will show its errors

Solution 5

The highest rated solution do not support 'strict=True', To solve the 'strict=True' not support issue, you can create a FakeRequest object to cheat RequestParser

class FakeRequest(dict):
    def __setattr__(self, name, value):
        object.__setattr__(self, name, value)

root_parser = reqparse.RequestParser()
root_parser.add_argument('id', type=int)
root_parser.add_argument('name', type=str)
root_parser.add_argument('nested_one', type=dict)
root_parser.add_argument('nested_two', type=dict)
root_args = root_parser.parse_args()

nested_one_parser = reqparse.RequestParser()
nested_one_parser.add_argument('id', type=int, location=('json',))

fake_request = FakeRequest()
setattr(fake_request, 'json', root_args['nested_one'])
setattr(fake_request, 'unparsed_arguments', {})

nested_one_args = nested_one_parser.parse_args(req=fake_request, strict=True)

fake_request = FakeRequest()
setattr(fake_request, 'json', root_args['nested_two'])
setattr(fake_request, 'unparsed_arguments', {})

nested_two_parser = reqparse.RequestParser()
nested_two_parser.add_argument('id', type=int, location=('json',))
nested_two_args = nested_two_parser.parse_args(req=fake_request, strict=True)

BTW: flask restful will rip RequestParser out, and replace it with Marshmallow Linkage

View more solutions