Access NVIDIA GPUs

Warning

This is an experimental feature and has not been extensively tested.

Prerequisites

To enable this feature, make sure you have a cuda-enabled GPU with the correct drivers installed on your manager host. Drivers are available from nvidia drivers page.

Note

Make sure you have no containers running on your machine before continuing.

Installing nvidia-container-runtime

Follow the steps here to install the nvidia-container-runtime. The same URL describes a few ways to register the runtime. For the nvidia containers to play nicely with MVI you should edit the daemon.json configuration file and set nvidia as the default runtime.

Restart docker:

>>> sudo systemctl daemon-reload  
>>> sudo systemctl restart docker  
At this point you can start a manager on this machine and use it like normal.

Deploying services with GPU access

Services will not automatically use the GPU even with nvidia-container-runtime enabled, for the GPU to be available you must first set an environment variable in .s2i/environment. Add the row:

NVIDIA_VISIBLE_DEVICES = all

to expose all GPUs to a service. Take a look the nvidia environment variables to see the available options. Now if you deploy the service with mvi deploy it will run in a container with access to cuda resources.