# Runtime Error: Trainer API Dataloader Using CPU but Expecting CUDA

**URL:** <https://discuss.huggingface.co/t/runtime-error-trainer-api-dataloader-using-cpu-but-expecting-cuda/38746>\
**Category:** 🤗Transformers\
**Created:** [May 5, 2023, 4:26am UTC](https://discuss.huggingface.co/t/runtime-error-trainer-api-dataloader-using-cpu-but-expecting-cuda/38746 "2023-05-05T04:26:50Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![tobyrm](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/tobyrm/32/15887_2.png) [@tobyrm](https://discuss.huggingface.co/u/tobyrm)\
**Post date:** [May 5, 2023, 4:26am UTC](https://discuss.huggingface.co/t/runtime-error-trainer-api-dataloader-using-cpu-but-expecting-cuda/38746/1 "2023-05-05T04:26:50Z")

</div>

I’m fine tuning a model like this:

```auto
            ds = datasets.Dataset.from_pandas(df_train[['text', 'label']])
            ds = ds.class_encode_column('label')
            ds = ds.train_test_split(test_size=0.2, stratify_by_column='label')
            ds1 = datasets.Dataset.from_pandas(df_test[['text', 'label']]).class_encode_column('label')
            ds = DatasetDict({
                'train': ds['train'],
                'val': ds['test'],
                'test': ds1,
            })

            data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
            tokenized_ds = ds.map(preprocess_function, batched=True)

            early_stop = EarlyStoppingCallback(early_stopping_patience=2)

            output_dir = bucket_dir + f'/llm/condition_models/{ __version__ }/{self.fingerprint}'
            os.makedirs(output_dir, exist_ok=True)

            # Train and store model
            training_args = TrainingArguments(
                output_dir=output_dir,
                learning_rate=2e-5,
                evaluation_strategy='epoch',
                save_strategy='epoch',
                per_device_train_batch_size=64,
                per_device_eval_batch_size=64,
                num_train_epochs=50,
                weight_decay=0.01,
                load_best_model_at_end=True,
                # dataloader_pin_memory=False,
            )
            trainer = Trainer(
                model=model,
                args=training_args,
                train_dataset=tokenized_ds["train"],
                eval_dataset=tokenized_ds["val"],
                tokenizer=tokenizer,
                data_collator=data_collator,
                callbacks=[early_stop],
            )
            trainer.train()

```

which produces the following error:

```auto
RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'

```

Indeed, when I check the device attribute in question:

```auto
trainer.get_train_dataloader().batch_sampler.sampler.generator.device

```

it shows that it is “cpu” despite CUDA being available and having `torch.set_default_tensor_type('torch.cuda.FloatTensor')` at the top of my module.

I tried overwriting the device on the generator and tried overwriting the sampler, but neither is allowed.

I am using `transformers==4.28.1` and `torch==2.0.0`.

I’m not sure where to go from here. Advice much appreciated.

---

<div class="post-metadata">

**Author:** ![inkognito1982](https://avatars.discourse-cdn.com/v4/letter/i/e9bcb4/32.png) [@inkognito1982](https://discuss.huggingface.co/u/inkognito1982)\
**Post date:** [December 12, 2023, 12:10pm UTC](https://discuss.huggingface.co/t/runtime-error-trainer-api-dataloader-using-cpu-but-expecting-cuda/38746/2 "2023-12-12T12:10:14Z")

</div>

Hi. I am having the same error as well while using the latest versions of transformers and trl library.  
I checked the trainer.get\_train\_dataloader().device and it shows as “cuda” but I get None for trainer.get\_train\_dataloader().generator .  
Not sure how to proceed.

```auto
RuntimeError Traceback (most recent call last)
<ipython-input-77-3435b262f1ae> in <cell line: 1>()
----> 1 trainer.train()

10 frames
/usr/local/lib/python3.10/dist-packages/torch/utils/_device.py in __torch_function__ (self, func, types, args, kwargs)
     75 if func in _device_constructors() and kwargs.get('device') is None:
     76 kwargs['device'] = self.device
---> 77 return func(*args, **kwargs)
     78 
     79 # NB: This is directly called from C++ in torch/csrc/Device.cpp

RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'

```

---

<div class="post-metadata">

**Author:** ![inkognito1982](https://avatars.discourse-cdn.com/v4/letter/i/e9bcb4/32.png) [@inkognito1982](https://discuss.huggingface.co/u/inkognito1982)\
**Post date:** [December 22, 2023, 9:42pm UTC](https://discuss.huggingface.co/t/runtime-error-trainer-api-dataloader-using-cpu-but-expecting-cuda/38746/3 "2023-12-22T21:42:35Z")

</div>

Ok. Finally figured it out. Just in the future, if anyone else stumbles upon this error, it usually happens if we set:

```auto
torch.set_default_device('cuda')

```