Addressing sys.excepthook error in bash script

17,152

Solution 1

I was seeing this error when piping output from a Python 2.6.2 script into the head command in bash on Ubuntu 9.04. I added try blocks to close stdout and stderr before exiting the script:

try:
    sys.stdout.close()
except:
    pass
try:
    sys.stderr.close()
except:
    pass

I am no longer seeing the error.

Solution 2

There are two steps you need to perform:

Step 1:

In your csvcut script, find all locations where sys.stdout.write() are called, make sure sys.stdout.flush() is called after each write().

Step 2:

With step 1 completed you should now be able to capture IOError within the Python script. Below is one example on how to handle broken pipe:

try:
    function_with_sys_stdout_write_call()
except IOError as e:
    # one example is broken pipe
    if e.strerror.lower() == 'broken pipe':
        exit(0)
    raise       # other real IOError

Hope it helps!

Solution 3

I assume that the csvcut Python script is otherwise functional but is throwing up an error when it tries to close files and exit.

If, as you say, the script is otherwise working and assuming that the error 'csvcut' is throwing up outputs to stderr then redirecting it to /dev/null would be a temp fix.

cat <<EOF >$OUTPUTFILE 2>/dev/null

Naturally any other error messages in your heredoc will also redirect there.

Share:
17,152
Jeff Severns Guntzel
Author by

Jeff Severns Guntzel

Updated on July 28, 2022

Comments

  • Jeff Severns Guntzel
    Jeff Severns Guntzel over 1 year

    I've written a bash script that is doing exactly what I want it to do, but kicking out the following error:

    close failed in file object destructor: sys.excepthook is missing lost sys.stderr

    I'm completely stumped on how to address this. Here is the script:

    #!/bin/bash
    
    usage () { echo "${0##*/} inputfile outputfile"; exit 1; }
    
    (($#==2)) || usage
    
    INPUTFILE="$1"
    OUTPUTFILE="$2"
    
    #  All that is written between between the 'cat' command and 
    #+ 'EOF' will be sent to the output file.
    cat <<EOF >$OUTPUTFILE
    $(date "+Generated on %m/%d/%y at %H:%M:%S")
    
    DATA AUDIT: $1
    
    ------------
    COLUMN NAMES
    ------------
    
    $(csvcut -n $INPUTFILE)
    
    ---------------------------------------
    FIRST TEN ROWS OF FIRST FIVE COLUMNS 
    ---------------------------------------
    
    $(csvcut -c 1,2,3,4,5 $INPUTFILE | head -n 10)
    
    ------------
    COLUMN STATS
    ------------
    
    $(csvcut $INPUTFILE | csvstat )
    
    ---END AUDIT
    EOF
    
    echo "Audited!"
    

    I am pretty new to shell scripts and very new to python. I would be grateful for any help.

  • Jeff Severns Guntzel
    Jeff Severns Guntzel over 12 years
    Even as a temp fix, that isn't working for me. Still getting the same error.
  • Snorfalorpagus
    Snorfalorpagus over 11 years
    Any explanation as to why this works? I've been getting this error in a cython script that does some redirection with dup/dup2 and this seems to fix things.
  • Admin
    Admin almost 9 years
    This was a proper fix in my case. Looks like a traceback was still being written to stderr while the script was quitting and flusing the stderr before quitting did the trick.
  • Aaron McDaid
    Aaron McDaid about 8 years
    The solution also works with calls to .flush instead of .close. I don't know if this means anything though. (close implies flush). Python 2.7.8 on Ubuntu 14.04
  • Aaron McDaid
    Aaron McDaid over 7 years
    The -u option to python (unbuffered output) seems relevant too. When -u is passed, I am unable to replicate the problem