Offloading LLM models to CPU uses only single core

rhwauiy89
Mon, 03 Jun 2024 06:13:26 +0000

I’m also having same problem with Mistral 7B, you can try BetterTransformer

Read full topic