Error/ Exception handling in for loop - python

19,322

Solution 1

I agree with ThatBird that wrapping too much code in a try-block can make debugging internal errors complicated. I would suggest utilizing python's continue keyword.

try:
    # smallest block of code you foresee an error in
    response = language_client.analyze_sentiment(document=document) # I think your exception is being raised in this call
except InvalidArgument as e:
    # your trace shows InvalidArgument being raised and it appears you dont care about it
    continue # continue to next iteration since this error is expected
except SomeOtherOkayException as e:
    # this is an example exception that is also OK and "skippable"
    continue # continue to next iteration
except Exception as e:
    # all other exceptions are BAD and unexpected.This is a larger problem than just this loop
    raise e # break the looping and raise to calling function

sentiment = response.document_sentiment
sentimentscore_list.append(sentiment.score)
magnitude_list.append(sentiment.magnitude)
# Add the description that was actually used to the description list
description_list.append(descr)
# more code here...

Essentially, you're explicitly catching Exceptions that are expected, and discarding that iteration if they occur and continuing to the next one. You should raise all other exceptions that are not expected.

Solution 2

In the traceback, look at the fourth line, it's the same line that is in your code and causing an exception. We always put try except around the code block that we think is going to cause an exception. Everything else is put outside the block.

try:
    response = language_client.analyze_sentiment(document=document)
except InvalidArgument:
    continue
# Assuming none of these would work if we don't get response?
description_list.append(descr)
sentiment = response.document_sentiment
entimentscore_list.append(sentiment.score)
magnitude_list.append(sentiment.magnitude)
# Add the description that was actually used to the description list

We try to get response from language client, it raises an exception saying InvalidArgument, we catch that. Now we know we don't need to do anything and we use continue, and move on to the next iteration.

You probably will need to import InvalidArgument like -

from google.api_core.exceptions import InvalidArgument

before using it in the code.

You are right about continue. More about continue statement and how to handle exceptions in python.

Share:
19,322
Christina
Author by

Christina

Updated on June 13, 2022

Comments

  • Christina
    Christina almost 2 years

    I am using the Google Cloud NL API to analyse the sentiment of some descriptions. As for some rows the error InvalidArgument: 400 The language vi is not supported for document_sentiment analysis.keeps popping up, I would like to build a way around it instead of desperately trying to find the reason why this happens and erase the responsible rows. Unfortunately, I am relatively new to Python and am not sure how to properly do it.

    My code is the following:

    description_list = []
    sentimentscore_list=[]
    magnitude_list=[]
    
    # Create a Language client
    language_client = google.cloud.language.LanguageServiceClient()
    
    
    for i in range(len(description)):      # use the translated description if the original description is not in English
        if description_trans[i] == '':
            descr = description[i]
        else:
            descr = description_trans[i]
    
    
        document = google.cloud.language.types.Document(
            content=descr,
            type=google.cloud.language.enums.Document.Type.PLAIN_TEXT)
    
        # Use Language to detect the sentiment of the text.
        response = language_client.analyze_sentiment(document=document)
        sentiment = response.document_sentiment
        sentimentscore_list.append(sentiment.score)
        magnitude_list.append(sentiment.magnitude)
        # Add the description that was actually used to the description list
        description_list.append(descr)
    

    Would anyone be able to explain me how to wrap this for loop (or probably the latter part is sufficient) into the error/exception handling so that it simply "skips over" the one it can't read and continues with the next one? Also I want the 'description_list' to be only appended when the description is actually analysed (so not when it gets stuck in the error handling).

    Any help is much appreciated!! Thanks :)


    Edit: I was asked for a more complete error traceback:

    Traceback (most recent call last):

      File "<ipython-input-64-6e3db1d976c9>", line 1, in <module>
        runfile('/Users/repos/NLPAnalysis/GoogleTest.py', wdir='/Users/repos/NLPAnalysis')
    
      File "/Users/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
        execfile(filename, namespace)
    
      File "/Users/anaconda3/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
        exec(compile(f.read(), filename, 'exec'), namespace)
    
      File "/Users/repos/NLPAnalysis/GoogleTest.py", line 45, in <module>
        response = language_client.analyze_sentiment(document=document)
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/cloud/language_v1/gapic/language_service_client.py", line 180, in analyze_sentiment
        return self._analyze_sentiment(request, retry=retry, timeout=timeout)
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py", line 139, in __call__
        return wrapped_func(*args, **kwargs)
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/api_core/retry.py", line 260, in retry_wrapped_func
        on_error=on_error,
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/api_core/retry.py", line 177, in retry_target
        return target()
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/api_core/timeout.py", line 206, in func_with_timeout
        return func(*args, **kwargs)
    
      File "/Users/anaconda3/lib/python3.6/site-packages/google/api_core/grpc_helpers.py", line 56, in error_remapped_callable
        six.raise_from(exceptions.from_grpc_error(exc), exc)
    
      File "<string>", line 3, in raise_from
    
    InvalidArgument: 400 The language vi is not supported for document_sentiment analysis.
    
  • Christina
    Christina over 5 years
    Thanks! I just added the traceback information. Would you still change anything based on this? Tbh it doesn't tell me much..
  • Christina
    Christina over 5 years
    Thanks for your response, it totally makes sense. However, I get a undefined name 'Invalid Argument' warning for the code. How do I get around this?
  • Christina
    Christina over 5 years
    Thanks, your answer makes a lot of sense! Is it correct to assume that when an exception is raised, everything that comes after the continue statement (line 6 - 9) are "skipped" for this loop and it directly jumps into the next one? I don't want the lists to be filled for this instance. Also, I get a undefined name 'Invalid Argument' warning for the code. How do I get around this?
  • Christina
    Christina over 5 years
    I just tried it by replacing InvalidArgument (as this did not work) with Exception as I thought it would at least also filter the InvalidArgument errors, but it did not filter anything. It just run through all for loop as if there was no exception raised which can't be right. Any ideas why this happened?
  • Christina
    Christina over 5 years
    I just tried it by replacing InvalidArgument (as this did not work) with Exception as I thought it would at least also filter the InvalidArgument errors, but it did not filter anything. It just run through all for loop as if there was no exception raised which can't be right. Any ideas why this happened?
  • Sushant
    Sushant over 5 years
    No exception must have raised. Why don't you try importing the InvalidArgument exception rather than using the broad exception term?
  • Dougyfresh
    Dougyfresh over 5 years
    You'll have to import it. I think its from here: from google.api_core.exceptions import InvalidArgument https://googlecloudplatform.github.io/google-cloud-python/la‌​test/core/exceptions‌​.html#google.api_cor‌​e.exceptions.Invalid‌​Argument The reason why it ran through as if there were no errors is because using Exception will catch/handle everything. Which is not what you want.