how to use CRF in tensorflow keras?

10,380

Solution 1

I run into a similar problem and spent a lot of time trying to get things to work. Here's what worked for me using python 3.6.5:

Seqeval:

pip install seqeval==0.0.5

Keras:

pip install keras==2.2.4

Keras-contrib (2.0.8):

git clone https://www.github.com/keras-team/keras-contrib.git

cd keras-contrib

python setup.py install

TensorFlow:

pip install tensorflow==1.14.0

Do pip list to make sure you have actually installed those versions (eg pip seqeval may automatically update your keras)

Then in your code import like so:

from keras.models import *
from keras.layers import LSTM, Embedding, Dense, TimeDistributed, Dropout, Bidirectional, Input
from keras_contrib.layers import CRF
#etc.

Hope this helps, good luck!

Solution 2

You can try tensorflow add-ons.(If you are using tensorflow version 2). You can try tf-crf-layer (if you are using tensorflow==1.15.0)

Share:
10,380

Related videos on Youtube

max yue
Author by

max yue

Updated on May 27, 2022

Comments

  • max yue
    max yue over 1 year

    The code is like this:

    import tensorflow as tf
    from keras_contrib.layers import CRF
    from tensorflow import keras
    
    def create_model(max_seq_len, adapter_size=64):
        """Creates a classification model."""
    
        # adapter_size = 64  # see - arXiv:1902.00751
    
        # create the bert layer
        with tf.io.gfile.GFile(bert_config_file, "r") as reader:
            bc = StockBertConfig.from_json_string(reader.read())
            bert_params = map_stock_config_to_params(bc)
            bert_params.adapter_size = adapter_size
            bert = BertModelLayer.from_params(bert_params, name="bert")
    
        input_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32', name="input_ids")
        # token_type_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32', name="token_type_ids")
        # output         = bert([input_ids, token_type_ids])
        bert_output = bert(input_ids)
        print("bert_output.shape: {}".format(bert_output.shape))  # (?, 100, 768)
    
        crf = CRF(len(tag2idx))
        logits = crf(bert_output)
        model = keras.Model(inputs=input_ids, outputs=logits)
        model.build(input_shape=(None, max_seq_len))
    
        # load the pre-trained model weights
        load_stock_weights(bert, bert_ckpt_file)
    
        # freeze weights if adapter-BERT is used
        if adapter_size is not None:
            freeze_bert_layers(bert)
    
        model.compile('adam', loss=crf.loss_function, metrics=[crf.accuracy])
    
        model.summary()
    
        return model
    

    I am using tensorflow keras and also use keras_contrib package, to do NER. it seems the tensorflow keras package does not work well with keras_contrib package.

    The Traceback information is listed below:

    Traceback (most recent call last):
      File "F:/_gitclone3/bert_examples/bert_ner_example_eval.py", line 120, in <module>
        model = create_model(max_seq_len, adapter_size=adapter_size)
      File "F:/_gitclone3/bert_examples/bert_ner_example_eval.py", line 101, in create_model
        logits = crf(bert_output)
      File "C:\Users\yuexiang\Anaconda3\lib\site-packages\keras\engine\base_layer.py", line 443, in __call__
        previous_mask = _collect_previous_mask(inputs)
      File "C:\Users\yuexiang\Anaconda3\lib\site-packages\keras\engine\base_layer.py", line 1311, in _collect_previous_mask
        mask = node.output_masks[tensor_index]
    AttributeError: 'Node' object has no attribute 'output_masks'
    

    How do I use CRF with tensorflow keras?

  • max yue
    max yue about 4 years
    Thank you. I mean using tensorflow.keras and crf, not keras and keras_contrib.crf. keras and keras_contrib.crf will work, but tensorflow.keras with keras_contrib.crf will not work. I am using bert-for-tf2 which uses tensorflow.keras not keras, so I want a crf package can work well with tensorflow.keras.
  • brogeo
    brogeo about 4 years
    That sounds to me like a completely different question from your original question. Anyway that error "AttributeError: 'Node' object has no attribute 'output_masks'" is apparently because of incompatible keras/keras contrib/tensorflow versions (stackoverflow.com/questions/51821537/…). So for now you probably have to play with the versions of keras/tensorflow and hopefully get something to work or wait for the developers to fix this issue.
  • marikamitsos
    marikamitsos over 3 years
    Welcome. Please edit your answer according to How do I write a good answer?.
  • Stanley Zheng
    Stanley Zheng over 1 year