Add GPU tests #2008
Description
🚀 Feature
Add GPU tests
Motivation
Several bugs have some up related to mismatching devices for the T5 Model. This is because we have no proper tests for running the model on a GPU or anything where there might be more than one device. The first step in this would be to add these GPU tests for running a T5Model.
Additional context
From @osalpekar on how to start this:
I think we would need to add a a new GitHub Actions workflow called test-linux-gpu.yml. Setting up the workflow should be the same as the existing CPU one, we'll just need to pass 3 more args (runner-type, gpu-arch-type, and gpu-arch-version) so that we run this on a GPU instance: https://github.com/pytorch/test-infra/wiki/Writing-generic-CI-jobs#simple-gpu. And in the script for the workflow, you can specify the pytest command for running those gpu tests