Mask modelling on specific words

vitali
Thu, 25 Mar 2021 04:17:51 +0000

to deal with vocabulary change, I had to (1) get vocab from the current model tokenizer tokenizer.get_vocab() (2) compare my custom vocab with the vocab of the model tokenizer (3) add my tokens to tokenizer vocab tokenizer.add_tokens(add_vocab) (4) resize the model for updated vocab model.resize_token_embeddings(len(tokenizer)). and cross my fingers that Trainer still works (would be very nice if Trainer could auto-reszise model for updated vocab, instead of crushing)

Read full topic