Access NVIDIA GPUs¶
This is an experimental feature and has not been extensively tested.
To enable this feature, make sure you have a cuda-enabled GPU with the correct drivers installed on your manager host. Drivers are available from nvidia drivers page.
Make sure you have no containers running on your machine before continuing.
Follow the steps here to install the nvidia-container-runtime. The same URL describes a few ways to register the runtime. For the nvidia containers to play nicely with Daeploy you should edit the daemon.json configuration file and set nvidia as the default runtime.
>>> sudo systemctl daemon-reload >>> sudo systemctl restart docker At this point you can start a manager on this machine and use it like normal.
Deploying services with GPU access¶
Services will not automatically use the GPU even with nvidia-container-runtime enabled, for the GPU to be available you must first set an environment variable in .s2i/environment. Add the row:
NVIDIA_VISIBLE_DEVICES = all
to expose all GPUs to a service. Take a look the
nvidia environment variables
to see the available options. Now if you deploy the service with
it will run in a container with access to cuda resources.