Looking for tool class to do predictions

PhilipMay
Fri, 09 Oct 2020 12:22:30 +0000

@stefan-it although I see no support for batching. At least not in the TextClassificationPipeline.

When I follow the naive approach:

model = AutoModelForSequenceClassification.from_pretrained(model_dir)
tokenizer = AutoTokenizer.from_pretrained(model_dir)
pipeline = TextClassificationPipeline(
    model=model,
    tokenizer=tokenizer,
    device=0,
)
pipeline(unlabeled_text_list)

I am getting an CUDA OOM if the list (unlabeled_text_list) is long…