github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.9.x/deploy-manage/manage/gpus.md (about)

     1  # Use GPUs
     2  
     3  Pachyderm currently supports GPUs through Kubernetes device plugins. If you
     4  already have a GPU enabled Kubernetes cluster through device plugins,
     5  skip to [Configure GPUs in Pipelines](#configure-gpus-in-pipelines).
     6  
     7  ## Set up a GPU-enabled Kubernetes Cluster
     8  
     9  For instructions on how to set up a GPU-enabled Kubernetes cluster
    10  through device plugins, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/).
    11  
    12  Depending your hardware and applications, setting up a GPU-enabled
    13  Kubernetes cluster might require significant effort. If you are run
    14  into issues, verify that the following issues are addressed:
    15  
    16  1. The correct software is installed on the GPU machines so
    17  that applications that run in Docker containers can use the GPUs. This is
    18  highly dependent on the manufacturer of the GPUs and how you use them.
    19  The most straightforward approach is to get a VM image with this
    20  pre-installed and use management software such as
    21  [kops nvidia-device-plugin](https://github.com/kubernetes/kops/tree/master/hooks/nvidia-device-plugin).
    22  
    23  2. Kubernetes exposes the GPU resources. You can check this by
    24  describing the GPU nodes with `kubectl describe node`. If the GPU resources
    25  available configured correctly, you should see them as available for scheduling.
    26  
    27  3. Your application can access and use the GPUs. This may be as simple as making
    28  shared libraries accessible by the application that runs in your container. You
    29  can  configure this by injecting environment variables into the Docker image or
    30  passing environment variables through the pipeline spec.
    31  
    32  ## Configure GPUs in Pipelines
    33  
    34  If you already have a GPU-enabled Kubernetes cluster through device plugins,
    35  then using GPUs in your pipelines is as simple as setting up a GPU resource
    36  limit with the type and number of GPUs. The following text is an example
    37  of a pipeline spec for a GPU-enabled pipeline:
    38  
    39  !!! example
    40      ```json hl_lines="12 13 14 15 16"
    41      {
    42        "pipeline": {
    43          "name": "train"
    44        },
    45        "transform": {
    46          "image": "acme/your-gpu-image",
    47          "cmd": [
    48            "python",
    49            "train.py"
    50          ],
    51        },
    52        "resource_limits": {
    53          "memory": "1024M",
    54          "gpu": {
    55            "type": "nvidia.com/gpu",
    56            "number": 1
    57          }
    58        },
    59        "inputs": {
    60          "pfs": {
    61            "repo": "data",
    62            "glob": "/*"
    63          }
    64        ]
    65      }
    66      ```