Sequence Classification -- Fine Tune?

AlanFeder
Wed, 27 Jan 2021 20:17:21 +0000

Hi,

I am new to Transformers/NLP.

I am trying to use Transformers for text classification. If I am not classifying to one of the pre-made GLUE benchmarks (and using my own use-case classes & texts), do I have to “fine-tune” the model? If I have 35k texts, and 2 labels (imbalanced – 98% vs 2%) can I just use the AutoModelforSequenceClassification to throw a softmax on the end of the transformer, and train that softmax? Or do I fine-tune the whole thing, using this tutorial?

Thanks! I am excited about better understanding the field and more effectively using the library.