How to use ast.literal_eval in a pandas dataframe and handle exceptions

Question

I have a dataframe with a column containing a tuple data as a string. Eg. '(5,6)'. I need to convert this to a tuple structure. One way of doing it is using the ast.literal_eval(). I am using it in this way.

df['Column'] = df['Column'].apply(ast.literal_eval)

Unfortunately, my data in this column contains empty strings also. The ast.literal_eval() is not able to handle this. I get this error.

SyntaxError: unexpected EOF while parsing

I am unsure if this is because it is unable to handle such a character. Based on my reading, I found that ast.literal_eval() works only in cases when a list, dict or tuple is there inside a string structure.

To overcome this I tried to create my own function and return an empty string if it raises an exception.

def literal_return(val):
    try:
        return ast.literal_eval(val)
    except ValueError:
        return (val)

df['Column2'] = df['Column'].apply(literal_return)

Even in this case, the same error pops up. How do we handle this. It would be great even if there is a way to ignore certain rows to apply the function and apply on the rest. Any help is appreciated.

Solution 1

I would do it simply requiring a string type from each entry:

from ast import literal_eval
df['column_2'] = df.column_1.apply(lambda x: literal_eval(str(x)))

If You need to advanced Exception handling, You could do, for example:

def f(x):
    try:
        return literal_eval(str(x))   
    except Exception as e:
        print(e)
        return []

df['column_2'] = df.column_1.apply(lambda x: f(x))   

Solution 2

This works when the function is changed to:

def literal_return(val):
    try:
        return ast.literal_eval(val)
    except (ValueError, SyntaxError) as e:
        return val