How to save traceback / sys.exc_info() values in a variable?

Question

I want to save the name of the error and the traceback details into a variable. Here's is my attempt.

import sys
try:
    try:
        print x
    except Exception, ex:
        raise NameError
except Exception, er:
    print "0", sys.exc_info()[0]
    print "1", sys.exc_info()[1]
    print "2", sys.exc_info()[2]

Output:

0 <type 'exceptions.NameError'>
1 
2 <traceback object at 0xbd5fc8>

Desired Output:

0 NameError
1
2 Traceback (most recent call last):
  File "exception.py", line 6, in <module>
    raise NameError

P.S. I know this can be done easily using the traceback module, but I want to know usage of sys.exc_info()[2] object here.

Solution 1

This is how I do it:

>>> import traceback
>>> try:
...   int('k')
... except:
...   var = traceback.format_exc()
... 
>>> print var
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
ValueError: invalid literal for int() with base 10: 'k'

You should however take a look at the traceback documentation, as you might find there more suitable methods, depending to how you want to process your variable afterwards...

Solution 2

sys.exc_info() returns a tuple with three values (type, value, traceback).

1. Here type gets the exception type of the Exception being handled
2. value is the arguments that are being passed to constructor of exception class
3. traceback contains the stack information like where the exception occurred etc.

For Example, In the following program

try:

    a = 1/0

except Exception,e:

    exc_tuple = sys.exc_info()

Now If we print the tuple the values will be this.

1. exc_tuple[0] value will be "ZeroDivisionError"
2. exc_tuple[1] value will be "integer division or modulo by zero" (String passed as parameter to the exception class)
3. exc_tuple[2] value will be "trackback object at (some memory address)"

The above details can also be fetched by simply printing the exception in string format.

print str(e)

Solution 3

Use traceback.extract_stack() if you want convenient access to module and function names and line numbers.

Use ''.join(traceback.format_stack()) if you just want a string that looks like the traceback.print_stack() output.

Notice that even with ''.join() you will get a multi-line string, since the elements of format_stack() contain \n. See output below.

Remember to import traceback.

Here's the output from traceback.extract_stack(). Formatting added for readability.

>>> traceback.extract_stack()
[
   ('<string>', 1, '<module>', None),
   ('C:\\Python\\lib\\idlelib\\run.py', 126, 'main', 'ret = method(*args, **kwargs)'),
   ('C:\\Python\\lib\\idlelib\\run.py', 353, 'runcode', 'exec(code, self.locals)'),
   ('<pyshell#1>', 1, '<module>', None)
]

Here's the output from ''.join(traceback.format_stack()). Formatting added for readability.

>>> ''.join(traceback.format_stack())
'  File "<string>", line 1, in <module>\n
   File "C:\\Python\\lib\\idlelib\\run.py", line 126, in main\n
       ret = method(*args, **kwargs)\n
   File "C:\\Python\\lib\\idlelib\\run.py", line 353, in runcode\n
       exec(code, self.locals)\n  File "<pyshell#2>", line 1, in <module>\n'

Solution 4

Be careful when you take the exception object or the traceback object out of the exception handler, since this causes circular references and gc.collect() will fail to collect. This appears to be of a particular problem in the ipython/jupyter notebook environment where the traceback object doesn't get cleared at the right time and even an explicit call to gc.collect() in finally section does nothing. And that's a huge problem if you have some huge objects that don't get their memory reclaimed because of that (e.g. CUDA out of memory exceptions that w/o this solution require a complete kernel restart to recover).

In general if you want to save the traceback object, you need to clear it from references to locals(), like so:

import sys, traceback, gc
type, val, tb = None, None, None
try:
    myfunc()
except:
    type, val, tb = sys.exc_info()
    traceback.clear_frames(tb)
# some cleanup code
gc.collect()
# and then use the tb:
if tb:
    raise type(val).with_traceback(tb)

In the case of jupyter notebook, you have to do that at the very least inside the exception handler:

try:
    myfunc()
except:
    type, val, tb = sys.exc_info()
    traceback.clear_frames(tb)
    raise type(val).with_traceback(tb)
finally:
    # cleanup code in here
    gc.collect()

Tested with python 3.7.

p.s. the problem with ipython or jupyter notebook env is that it has %tb magic which saves the traceback and makes it available at any point later. And as a result any locals() in all frames participating in the traceback will not be freed until the notebook exits or another exception will overwrite the previously stored backtrace. This is very problematic. It should not store the traceback w/o cleaning its frames. Fix submitted here.

Solution 5

The object can be used as a parameter in Exception.with_traceback() function:

except Exception as e:
    tb = sys.exc_info()
    print(e.with_traceback(tb[2]))

View more solutions

Comments

[zmbq]

Did you try printing sys.exc_info()[x].__str__()?

[mac]

You might have misunderstood what is going on in your program: what you refer to as "sys.exc_info()[2] object" is an instance of the traceback object (=you are using the traceback module already). Now, you can manipulate that object without using the helper functions in the traceback module, but that doesn't change the fact that you are still using it. :)

[codersofthedark]

So @mac please help me using accessing the value from this object with or without using the helper function.

[mac]

@dragosrsupercool - As I mentioned in my answer below, you should look at the traceback documentation. I provided an example of how retrieve the data textually, but there are other methods of the object that allow you to extract the exception name, the row of the code, etc... the right one really depends on how you want to manipulate the value afterwards...

[codersofthedark]

acutally I did read traceback documentation and its pretty working when I use traceback module directly.. But when I use sys.exc_info()[2] which is a afcourse a traceback class object, I am not able to use those same function here.. something like sys.exc_info()[2].tb_text doesnt work.. . any idea why?

[pythonlarry]

My answer to another question may help illustrate the details - with links! For canned strings, standard library traceback module seems okay. If you want to get the details, read the source (<python install path>/Lib/traceback.py) for more info.

[sivabudh]

See if my answer here can help you: stackoverflow.com/a/38410138/65313

[codersofthedark]

I was seeking for a method without using traceback module. Is there someway we can just print trace-back details from this object reference? sys.exc_info()[2]

[codersofthedark]

Right, that is why I thought we can do something like sys.exc_info()[2].format_exc(), but this dont work.. Thus, I wonder how can I extract value from trace-back object sys.exc_info()[2]. Any idea?

[mac]

@dragosrsupercool - "that is why I thought we can do something like sys.exc_info()[2].format_exc()" that makes no sense. You are trying to call a traceback module function as if it were a method of the traceback object...

[codersofthedark]

sys.exc_info()[2].tb_text gives follow error -> AttributeError: 'traceback' object has no attribute 'tb_text'

[mac]

@dragosrsupercool - sys.exc_info()[2].tb_frame.f_code.co_names[3], but it make no sense whatsoever... If there is a module called traceback in the standard library, there is a reason for it... :)

[codersofthedark]

sys.exc_info()[2].tb_frame.f_code.co_names[3] prints 'ex' and not the exception ... sys.exc_info()[2] is there for a reason too ... I want to know how can I make use of it ...

[wizzwizz4]

@codersofthedark traceback.format_exception(*sys.exc_info()) is the way to do it. But that's functionally equivalent to traceback.format_exc().

[Jinghao Shi]

For Python3, exc_tuple[1] (aka value) is the instance of the exception, not the "String passed as parameter". See: docs.python.org/3/library/sys.html#sys.exc_info

[Henrik]

Shouldn't it be "except Exception as e:"?