How to get Python exception text

Question

I want to embed python in my C++ application. I'm using Boost library - great tool. But i have one problem.

If python function throws an exception, i want to catch it and print error in my application or get some detailed information like line number in python script that caused error.

How can i do it? I can't find any functions to get detailed exception information in Python API or Boost.

try {
module=import("MyModule"); //this line will throw excetion if MyModule contains an   error
} catch ( error_already_set const & ) {
//Here i can said that i have error, but i cant determine what caused an error
std::cout << "error!" << std::endl;
}

PyErr_Print() just prints error text to stderr and clears error so it can't be solution

Solution 1

Well, I found out how to do it.

Without boost (only error message, because code to extract info from traceback is too heavy to post it here):

PyObject *ptype, *pvalue, *ptraceback;
PyErr_Fetch(&ptype, &pvalue, &ptraceback);
//pvalue contains error message
//ptraceback contains stack snapshot and many other information
//(see python traceback structure)

//Get error message
char *pStrErrorMessage = PyString_AsString(pvalue);

And BOOST version

try{
//some code that throws an error
}catch(error_already_set &){

    PyObject *ptype, *pvalue, *ptraceback;
    PyErr_Fetch(&ptype, &pvalue, &ptraceback);

    handle<> hType(ptype);
    object extype(hType);
    handle<> hTraceback(ptraceback);
    object traceback(hTraceback);

    //Extract error message
    string strErrorMessage = extract<string>(pvalue);

    //Extract line number (top entry of call stack)
    // if you want to extract another levels of call stack
    // also process traceback.attr("tb_next") recurently
    long lineno = extract<long> (traceback.attr("tb_lineno"));
    string filename = extract<string>(traceback.attr("tb_frame").attr("f_code").attr("co_filename"));
    string funcname = extract<string>(traceback.attr("tb_frame").attr("f_code").attr("co_name"));
... //cleanup here

Solution 2

This is the most robust method I've been able to come up so far:

    try {
        ...
    }
    catch (bp::error_already_set) {
        if (PyErr_Occurred()) {
            msg = handle_pyerror(); 
        }
        py_exception = true;
        bp::handle_exception();
        PyErr_Clear();
    }
    if (py_exception) 
    ....


// decode a Python exception into a string
std::string handle_pyerror()
{
    using namespace boost::python;
    using namespace boost;

    PyObject *exc,*val,*tb;
    object formatted_list, formatted;
    PyErr_Fetch(&exc,&val,&tb);
    handle<> hexc(exc),hval(allow_null(val)),htb(allow_null(tb)); 
    object traceback(import("traceback"));
    if (!tb) {
        object format_exception_only(traceback.attr("format_exception_only"));
        formatted_list = format_exception_only(hexc,hval);
    } else {
        object format_exception(traceback.attr("format_exception"));
        formatted_list = format_exception(hexc,hval,htb);
    }
    formatted = str("\n").join(formatted_list);
    return extract<std::string>(formatted);
}

Solution 3

In the Python C API, PyObject_Str returns a new reference to a Python string object with the string form of the Python object you're passing as the argument -- just like str(o) in Python code. Note that the exception object does not have "information like line number" -- that's in the traceback object (you can use PyErr_Fetch to get both the exception object and the traceback object). Don't know what (if anything) Boost provides to make these specific C API functions easier to use, but, worst case, you could always resort to these functions as they are offered in the C API itself.

Solution 4

This thread has been very useful for me, but I had problems with the Python C API when I tried to extract the error message itself with no traceback. I found plenty of ways to do that in Python, but I couldn't find any way to do this in C++. I finally came up with the following version, which uses the C API as little as possible and instead relies much more on boost python.

PyErr_Print();

using namespace boost::python;

exec("import traceback, sys", mainNamespace_);
auto pyErr = eval("str(sys.last_value)", mainNamespace_);
auto pyStackTrace = eval("'\\n'.join(traceback.format_exception(sys.last_type, sys.last_value, sys.last_traceback))", mainNamespace_);

stackTraceString_ = extract<std::string>(pyStackTrace);
errorSummary_ = extract<std::string>(pyErr);

The reason this works is because PyErr_Print() also sets the value for sys.last_value, sys.last_type, and sys.last_traceback. Those are set to the same values as sys.exc_info would give, so this is functionally similar to the following python code:

import traceback
import sys

try:
    raise RuntimeError("This is a test")
except:
    err_type = sys.exc_info()[0]
    value = sys.exc_info()[1]
    tb = sys.exc_info()[2]

    stack_trace = "\n".join(traceback.format_exception(err_type, value, tb))
    error_summary = str(value)


print(stack_trace)
print(error_summary)

I hope someone finds this useful!

Solution 5

Here's some code based on some of the other answers and comments, nicely formatted with modern C++ and comments. Minimally tested but it seems to work.

#include <string>
#include <boost/python.hpp>
#include <Python.h>

// Return the current Python error and backtrace as a string, or throw
// an exception if there was none.
std::string python_error_string() {
  using namespace boost::python;

  PyObject* ptype = nullptr;
  PyObject* pvalue = nullptr;
  PyObject* ptraceback = nullptr;

  // Fetch the exception information. If there was no error ptype will be set
  // to null. The other two values might set to null anyway.
  PyErr_Fetch(&ptype, &pvalue, &ptraceback);
  if (ptype == nullptr) {
    throw std::runtime_error("A Python error was detected but when we called "
                             "PyErr_Fetch() it returned null indicating that "
                             "there was no error.");
  }

  // Sometimes pvalue is not an instance of ptype. This converts it. It's
  // done lazily for performance reasons.
  PyErr_NormalizeException(&ptype, &pvalue, &ptraceback);
  if (ptraceback != nullptr) {
    PyException_SetTraceback(pvalue, ptraceback);
  }

  // Get Boost handles to the Python objects so we get an easier API.
  handle<> htype(ptype);
  handle<> hvalue(allow_null(pvalue));
  handle<> htraceback(allow_null(ptraceback));

  // Import the `traceback` module and use it to format the exception.
  object traceback = import("traceback");
  object format_exception = traceback.attr("format_exception");
  object formatted_list = format_exception(htype, hvalue, htraceback);
  object formatted = str("\n").join(formatted_list);
  return extract<std::string>(formatted);
}

Btw I was curious why everyone is using handle<> instead of handle. Apparently it disables template argument deduction. Not sure why you'd want that here but it isn't the same anyway, and the Boost docs say to use handle<> too so I guess there is a good reason.

View more solutions

Comments

[Anton Kiselev]

thanks a lot, Alex. I was looking a way to make it without direct calling of PyAPI - i thougth Boost can deal with exceptions, but Boost can't :(

[Alex Martelli]

@Anton, glad I helped, so what about upvoting and accepting this answer?-) Use the checkmark icon under the number of upvotes for this answer (currently 0;-).

[Kyle C]

Awesome, this is exactly what I have been looking for... works great.

[uckelman]

It's apparently ok to pass an empty handle to format_exception, so you don't need the !tb case.

[D. A.]

This is nice. I've discovered in some cases (for me, a boost;:python::import of something not in my PYTHONPATH) ptraceback will be 0, so I'd protect against usage of a ptraceback if it is 0. Also, can you comment on what we can do with extype? I suppose printing the text of the python exception type is meaningful. How do we do that?

[elmo]

One additional question: aren't we leaking memory in above? What frees objects returned by PyErr_Fetch? (I am not sure about both CPython and boost::pythoon cases)

[solublefish]

I would sure be interested in that non-boost code to extract the traceback. Or just some description of the structure, for which I can't seem to find documentation.

[DJMcMayhem]

This solution works great, bu you will need to call PyErr_NormalizeException(&exc, &val, &tb); like this answer says.

[m. c.]

According to doc, ptype pvalue ptraceback can be have no content, null, even the pyobject pointer itself is not null.

[Mark Ransom]

And don't forget PyUnicode_AsWideCharString or something similar to turn the returned object from PyObject_Str to a useful C-string.

[aCuria]

How do you initialize mainNamespace_ here?

[Daniel Hesslow]

import("main").attr("dict") seems to do the trick for me.

[user3875388]

Why my PyString_AsString(pvalue); return nullptr? And PyString_AsString(PyObject_Str(pValue)); did the right thing.

[Janek_Kozicki]

Thanks, this worked for me: gitlab.com/yade-dev/trunk/-/merge_requests/581

[Timmmm]

@elmo It does not leak memory because PyErr_Fetch() returns references to existing objects, it doesn't allocate new ones. Unfortunately the only way to know that with C is to read the source code because it is rarely documented. One of the things that Rust fixes.