RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

jfschaeper
Tue, 05 Sep 2023 07:20:41 +0000

I got the exact same error with my model. Also using deepspeed and bitsandbytes. Did you manage to solve the issue?
My code runs fine on a single gpu, but fails on multiple. (using A100)