sigs.k8s.io/kueue@v0.6.2/site/content/en/docs/tasks/integrate_a_custom_job.md (about)

     1  ---
     2  title: "Integrate a custom Job with Kueue"
     3  date: 2023-07-25
     4  weight: 8
     5  description: >
     6    Integrate a custom Job with Kueue.
     7  ---
     8  
     9  Kueue integrates with a couple of Jobs, including Kubernetes batch Job, MPIJob, RayJob or JobSet.  
    10  
    11  There are two options for integrating a Job-like CRD with Kueue:
    12  - As part of the Kueue repository
    13  - Writing an external controller
    14  
    15  This guide is for [platform developers](/docs/tasks#platform-developer) and focuses on the first approach.
    16  If there is a widely used Job which you would like supported in Kueue, please consider contributing.
    17  This page shows how to integrate a custom Job with Kueue.
    18  
    19  ## Pre-requisites
    20  
    21  1. The custom Job CRD should have a suspend-like field, with semantics similar to the [`suspend` field in a Kubernetes Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/#suspending-a-job).
    22  
    23  ## What you need to specify
    24  
    25  1. Configuration
    26  2. Controller
    27  3. Webhook
    28  4. Adjust build system
    29  
    30  ## Configuration
    31  
    32  Add your framework name to `.integrations.frameworks` in [controller_manager_config.yaml](https://github.com/kubernetes-sigs/kueue/blob/main/config/components/manager/controller_manager_config.yaml)
    33  
    34  ## Controller
    35  
    36  Add a new folder in `./pkg/controller/jobs/` to host the implementation of the integration.
    37  
    38  1. Register the new framework in `func Init()`
    39  2. Add RBAC Authorization using [kubebuilder marker comments](https://book.kubebuilder.io/reference/markers/rbac.html)
    40      - Add RBAC Authorization for priorityclasses, events, workloads, resourceflavors and any resources you want.
    41  3. Implement the `GenericJob` interface, and other optional [interfaces defined by the framework](https://github.com/kubernetes-sigs/kueue/blob/main/pkg/controller/jobframework/interface.go).
    42  
    43  ## Webhook
    44  
    45  Create a webhook file in the dedicated directory for the integration.
    46  
    47  You can learn how to create webhook in this [page](https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html).
    48  
    49  The framework provides some validation functions for common labels and annotations.
    50  
    51  Your webhook should have ability to suspend jobs.
    52  
    53  
    54  ## Adjust build system
    55  
    56  1. Add required dependencies to compile the controller and webhook code. For example, using `go get github.com/kubeflow/mpi-operator@0.4.0`.
    57  2. Update [Makefile](https://github.com/kubernetes-sigs/kueue/blob/main/Makefile) for testing.
    58     - Add commands which copy the crd of your custom job to the Kueue project.
    59     - Add your custom job operator crd dependencies into `test-integration`.
    60  
    61     For testing files, you can check the sample test files in [completed integrations](#completed-integrations) to learn how to implement them.
    62  
    63  ## Completed integrations
    64  Here are completed integrations you can learn from:
    65     - [BatchJob](https://github.com/kubernetes-sigs/kueue/tree/main/pkg/controller/jobs/job)
    66     - [JobSet](https://github.com/kubernetes-sigs/kueue/tree/main/pkg/controller/jobs/jobset)
    67     - [MPIJob](https://github.com/kubernetes-sigs/kueue/tree/main/pkg/controller/jobs/mpijob)
    68     - [RayJob](https://github.com/kubernetes-sigs/kueue/tree/main/pkg/controller/jobs/rayjob)