Error: AttributeError: module 'transformers' has no attribute 'TFBertModel'

Question

I am applying transfer learning with the python framework (PyTorch). I am getting the below error, when loading a PyTorch pre-trained model in Google Colab. After changing the code 1 to be as code 2, I got the same error.

CODE 1:  BertModel.from_pretrained
CODE 2: TFBertModel.from_pretrained
Error: AttributeError: module 'transformers' has no attribute 'TFBertModel'

I tried to search the internet, but I didn't find any useful content.

Solution

You should probably list the available package with its version in your python and your Colab link, for TFBertModel is only available when you have tensorflow.

In order to reproduce your error. I play around in the Colab as following:

1. No tensorflow cause error when you import TFBertModel

!pip install transformers
from transformers import BertModel, TFBertModel # no attribute 'TFBertModel'
!pip install tensorflow-gpu
from transformers import BertModel, TFBertModel # good to go

2. Directly use BertModel

!pip install transformers
from transformers import BertModel
BertModel.from_pretrained # good to go

As the result of my testing, you should probably check out if you import the TFBertModel while let tensorflow uninstalled.

Transformers under the master branch import the TFBertModel only if is_tf_available() is set to True. Here is the code for if_is_tf_available():

# transformers/src/transformers/file_utils.py 
# >>> 107 lines
def is_tf_available():
    return _tf_available

# >>> 48 lines
try:
    USE_TF = os.environ.get("USE_TF", "AUTO").upper()
    USE_TORCH = os.environ.get("USE_TORCH", "AUTO").upper()

    if USE_TF in ("1", "ON", "YES", "AUTO") and USE_TORCH not in ("1", "ON", "YES"):
        import tensorflow as tf

        assert hasattr(tf, "__version__") and int(tf.__version__[0]) >= 2
        _tf_available = True  # pylint: disable=invalid-name
        logger.info("TensorFlow version {} available.".format(tf.__version__))
    else:
        logger.info("Disabling Tensorflow because USE_TORCH is set")
        _tf_available = False
except (ImportError, AssertionError):
    _tf_available = False  # pylint: disable=invalid-name

Comments

[nbro]

Hi. You probably should provide the link to the Colab notebook with all your code.

[user108569]

are you using tensorflow 2.0?