General decorator to wrap try except in python?
Solution 1
You could use a defaultdict and the context manager approach as outlined in Raymond Hettinger's PyCon 2013 presentation
from collections import defaultdict
from contextlib import contextmanager
@contextmanager
def ignored(*exceptions):
try:
yield
except exceptions:
pass
item = defaultdict(str)
obj = dict()
with ignored(Exception):
item['a'] = obj.get(2).get(3)
print item['a']
obj[2] = dict()
obj[2][3] = 4
with ignored(Exception):
item['a'] = obj.get(2).get(3)
print item['a']
Solution 2
There are lots of good answers here, but I didn't see any that address the question of whether you can accomplish this via decorators.
The short answer is "no," at least not without structural changes to your code. Decorators operate at the function level, not on individual statements. Therefore, in order to use decorators, you would need to move each of the statements to be decorated into its own function.
But note that you can't just put the assignment itself inside the decorated function. You need to return the rhs expression (the value to be assigned) from the decorated function, then do the assignment outside.
To put this in terms of your example code, one might write code with the following pattern:
@return_on_failure('')
def computeA():
item['a'] = myobject.get('key').METHOD_THAT_DOESNT_EXIST()
item["a"] = computeA()
return_on_failure
could be something like:
def return_on_failure(value):
def decorate(f):
def applicator(*args, **kwargs):
try:
return f(*args,**kwargs)
except:
print('Error')
return value
return applicator
return decorate
Solution 3
It's very easy to achieve using configurable decorator.
def get_decorator(errors=(Exception, ), default_value=''):
def decorator(func):
def new_func(*args, **kwargs):
try:
return func(*args, **kwargs)
except errors, e:
print "Got error! ", repr(e)
return default_value
return new_func
return decorator
f = get_decorator((KeyError, NameError), default_value='default')
a = {}
@f
def example1(a):
return a['b']
@f
def example2(a):
return doesnt_exist()
print example1(a)
print example2(a)
Just pass to get_decorator tuples with error types which you want to silence and default value to return. Output will be
Got error! KeyError('b',)
default
Got error! NameError("global name 'doesnt_exist' is not defined",)
default
Edit: Thanks to martineau i changed default value of errors to tuples with basic Exception to prevents errors.
Solution 4
It depends on what exceptions you expect.
If your only use case is get()
, you could do
item['b'] = myobject.get('key2', '')
For the other cases, your decorator approach might be useful, but not in the way you do it.
I'll try to show you:
def f(func):
def silenceit(*args, **kwargs): # takes all kinds of arguments
try:
return func(*args, **kwargs) # returns func's result
except Exeption, e:
print('Error:', e)
return e # not the best way, maybe we'd better return None
# or a wrapper object containing e.
return silenceit # on the correct level
Nevertheless, f(some_undefined_function())
won't work, because
a) f()
isn't yet active at the executon time and
b) it is used wrong. The right way would be to wrap the function and then call it: f(function_to_wrap)()
.
A "layer of lambda" would help here:
wrapped_f = f(lambda: my_function())
wraps a lambda function which in turn calls a non-existing function. Calling wrapped_f()
leads to calling the wrapper which calls the lambda which tries to call my_function()
. If this doesn't exist, the lambda raises an exception which is caught by the wrapper.
This works because the name my_function
is not executed at the time the lambda is defined, but when it is executed. And this execution is protected and wrapped by the function f()
then. So the exception occurs inside the lambda and is propagated to the wrapping function provided by the decorator, which handles it gracefully.
This move towards inside the lambda function doesn't work if you try to replace the lambda function with a wrapper like
g = lambda function: lambda *a, **k: function(*a, **k)
followed by a
f(g(my_function))(arguments)
because here the name resolution is "back at the surface": my_function
cannot be resolved and this happens before g()
or even f()
are called. So it doesn't work.
And if you try to do something like
g(print)(x.get('fail'))
it cannot work as well if you have no x
, because g()
protects print
, not x
.
If you want to protect x
here, you'll have to do
value = f(lambda: x.get('fail'))
because the wrapper provided by f()
calls that lambda function which raises an exception which is then silenced.
Solution 5
in your case you first evaluate the value of the meow call (which doesn't exist) and then wrap it in the decorator. this doesn't work that way.
first the exception is raised before it was wrapped, then the wrapper is wrongly indented (silenceit
should not return itself). You might want to do something like:
def hardfail():
return meow() # meow doesn't exist
def f(func):
def wrapper():
try:
func()
except:
print 'error'
return wrapper
softfail =f(hardfail)
output:
>>> softfail()
error
>>> hardfail()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in hardfail
NameError: global name 'meow' is not defined
anyway in your case I don't understand why you don't use a simple method such as
def get_subkey(obj, key, subkey):
try:
return obj.get(key).get(subkey, '')
except AttributeError:
return ''
and in the code:
item['a'] = get_subkey(myobject, 'key', 'subkey')
Edited:
In case you want something that will work at any depth. You can do something like:
def get_from_object(obj, *keys):
try:
value = obj
for k in keys:
value = value.get(k)
return value
except AttributeError:
return ''
That you'd call:
>>> d = {1:{2:{3:{4:5}}}}
>>> get_from_object(d, 1, 2, 3, 4)
5
>>> get_from_object(d, 1, 2, 7)
''
>>> get_from_object(d, 1, 2, 3, 4, 5, 6, 7)
''
>>> get_from_object(d, 1, 2, 3)
{4: 5}
And using your code
item['a'] = get_from_object(obj, 2, 3)
By the way, on a personal point of view I also like @cravoori solution using contextmanager. But this would mean having three lines of code each time:
item['a'] = ''
with ignored(AttributeError):
item['a'] = obj.get(2).get(3)
Mittenchops
Updated on July 05, 2022Comments
-
Mittenchops almost 2 years
I'd interacting with a lot of deeply nested json I didn't write, and would like to make my python script more 'forgiving' to invalid input. I find myself writing involved try-except blocks, and would rather just wrap the dubious function up.
I understand it's a bad policy to swallow exceptions, but I'd rather prefer they to be printed and analysed later, than to actually stop execution. It's more valuable, in my use-case to continue executing over the loop than to get all keys.
Here's what I'm doing now:
try: item['a'] = myobject.get('key').METHOD_THAT_DOESNT_EXIST() except: item['a'] = '' try: item['b'] = OBJECT_THAT_DOESNT_EXIST.get('key2') except: item['b'] = '' try: item['c'] = func1(ARGUMENT_THAT_DOESNT_EXIST) except: item['c'] = '' ... try: item['z'] = FUNCTION_THAT_DOESNT_EXIST(myobject.method()) except: item['z'] = ''
Here's what I'd like, (1):
item['a'] = f(myobject.get('key').get('subkey')) item['b'] = f(myobject.get('key2')) item['c'] = f(func1(myobject) ...
or (2):
@f def get_stuff(): item={} item['a'] = myobject.get('key').get('subkey') item['b'] = myobject.get('key2') item['c'] = func1(myobject) ... return(item)
...where I can wrap either the single data item (1), or a master function (2), in some function that turns execution-halting exceptions into empty fields, printed to stdout. The former would be sort of an item-wise skip - where that key isn't available, it logs blank and moves on - the latter is a row-skip, where if any of the fields don't work, the entire record is skipped.
My understanding is that some kind of wrapper should be able to fix this. Here's what I tried, with a wrapper:
def f(func): def silenceit(): try: func(*args,**kwargs) except: print('Error') return(silenceit)
Here's why it doesn't work. Call a function that doesn't exist, it doesn't try-catch it away:
>>> f(meow()) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'meow' is not defined
Before I even add a blank return value, I'd like to get it to try-catch correctly. If the function had worked, this would have printed "Error", right?
Is a wrapper function the correct approach here?
UPDATE
I've had a lot of really useful, helpful answers below, and thank you for them---but I've edited the examples I used above to illustrate that I'm trying to catch more than nested key errors, that I'm looking specifically for a function that wraps a try-catch for...
- When a method doesn't exist.
- When an object doesn't exist, and is getting a method called on it.
- When an object that does not exist is being called as an argument to a function.
- Any combination of any of these things.
- Bonus, when a function doesn't exist.
-
Mittenchops over 11 yearsI'm trying to make it more generic, not just for one-level deep missing keys, but with a lot of functions that can fail to assign data for a lot of reasons.
-
Mittenchops over 11 yearsThanks! The return in the wrong place fixes what I thought I was doing, but wasn't. But I'm still at a loss in how it the internal exception from meow() is called before the wrap. The reason I can't use the simple method is that I am calling at several different depths, using several different objects with different attributes. I'd be writing a function per assignment, which would be as cumbersome as just try-catching. So, I'm looking for something that can generically catch failed functions and return '', printing an error to stdout.
-
Mittenchops over 11 yearsThis was helpful, but I'm still working on it. (Exception is misspelled above, but got it.) Taking this a little farther, is there a way that, instead of wrapping with lambda, I could make another function, g() that does that lambda wrap for me on a generic function, including *args, and **kwargs? So that I could call g(anyfunction()) and it would have the full effect I'm looking for?
-
Mittenchops over 11 yearsHmm, that's neat. I'm going to investigate this. Thanks.
-
Mittenchops over 11 yearsWhen I tried it, I did
>>> wrapped_p = f(lambda *z,**z: func(*z,**z))
and gotFile "<stdin>", line 1 SyntaxError: duplicate argument 'z' in function definition
-
glglgl over 11 yearsOf course; you could do a
g = lambda function: lambda *a, **k: function(*a, **k)
which enables you to dog(anyfunction)()
. (Only in this way: you don't want to wrap the function result, but the function itself.) -
Mittenchops over 11 yearsThanks, that was silly of me. But even using this function to wrap my function, I seem to have the same problem. So,
>>> g = f(lambda function: lambda *a, **k: function(*a,**k))
Then I call it with>>> g(print)(x.get('fail'))
and get a message that x doesn't exist, rather than the silent failure I'm looking for. -
glglgl over 11 yearsPhew, of course. That was my fault. In this context it cannot work because of the time where name resolution occurs. It has to be moved away from immediate execution. I'll elaborate in my answer above.
-
jdennison almost 11 yearslink is down. any ideas where to find it?
-
Peter Wood over 9 yearsThat's a great talk, really brought a lot of things together for me. I'll now recommend this along with Code Like a Pythonista: Idiomatic Python. Thanks.
-
alper about 4 yearsIn
ignored
function instead of usingpass
can I useraise
and catch an exception on the main call in order to print an error message for the related code block? @iruvar -
iruvar about 4 years@alper, that would kind of defeat the point of using
ignored
, wouldn't it. If I understand your question you seek to print details correspond to the exception at hand. You could do this instead -import sys, traceback
and then addex_type, ex, tb = sys.exc_info()
followed bytraceback.print_tb(tb)
just before thepass
inignored