sigs.k8s.io/kueue@v0.6.2/site/content/en/docs/concepts/resource_flavor.md (about)

     1  ---
     2  title: "Resource Flavor"
     3  date: 2022-03-14
     4  weight: 2
     5  description: >
     6    An object that you can define to describe what resources are available
     7    in a cluster.
     8  ---
     9  
    10  Resources in a cluster are typically not homogeneous. Resources could differ in:
    11  
    12  - pricing and availability (for example, spot verus on-demand VMs)
    13  - architecture (for example, x86 versus ARM CPUs)
    14  - brands and models (for example, Radeon 7000 versus Nvidia A100 versus T4 GPUs)
    15  
    16  A ResourceFlavor is an object that represents these resource variations and
    17  allows you to associate them with cluster nodes through labels, taints and tolerations.
    18  
    19  **Note**: If the resources in your cluster are homogeneous, you can use 
    20  an [empty ResourceFlavor](#empty-resourceflavor) instead of adding labels to custom ResourceFlavors.
    21  
    22  A sample ResourceFlavor looks like the following:
    23  
    24  ```yaml
    25  apiVersion: kueue.x-k8s.io/v1beta1
    26  kind: ResourceFlavor
    27  metadata:
    28    name: "spot"
    29  spec:
    30    nodeLabels:
    31      instance-type: spot
    32    nodeTaints:
    33    - effect: NoSchedule
    34      key: spot
    35      value: "true"
    36    tolerations:
    37    - key: "spot-taint"
    38      operator: "Exists"
    39      effect: "NoSchedule"
    40  ```
    41  
    42  You can use the `.metadata.name` field to reference a ResourceFlavor from a
    43  [ClusterQueue](/docs/concepts/cluster_queue) in the `.spec.resourceGroups[*].flavors[*].name` field.
    44  
    45  ## ResourceFlavor labels
    46  
    47  **Requires Kubernetes 1.23 or newer**
    48  
    49  To associate a ResourceFlavor with a subset of nodes of your cluster, you can
    50  configure the `.spec.nodeLabels` field with matching node labels that uniquely identify
    51  the nodes. If you are using [cluster autoscaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler)
    52  (or equivalent controllers), make sure that the controller is configured to add those labels when
    53  adding new nodes.
    54  
    55  To guarantee that the Pods in the [Workload](/docs/concepts/workload) run on the nodes associated to the flavor
    56  that Kueue selected, Kueue performs the following steps:
    57  
    58  1. When admitting a Workload, Kueue evaluates the
    59     [`.nodeSelector`](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector)
    60     and [`.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution`](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity)
    61     fields in the PodSpecs of your [Workload](/docs/concepts/workload) against the
    62     ResourceFlavor labels.
    63     `ResourceFlavors` that don't match the node affinity of the Workload
    64     cannot be assigned to a Workload's podSet.
    65  
    66  
    67  2. Once the Workload is admitted:
    68     - Kueue adds the ResourceFlavor labels to the
    69      `.nodeSelector` of the underlying Workload Pod templates. This occurs if the Workload
    70       didn't specify the `ResourceFlavor` labels already as part of its nodeSelector.
    71  
    72       For example, for a [batch/v1.Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/),
    73       Kueue adds the labels to the `.spec.template.spec.nodeSelector` field. This
    74       guarantees that the workload's Pods can only be scheduled on the nodes
    75       targeted by the flavor that Kueue assigned to the Workload.
    76     
    77     - Kueue adds the tolerations to the underlying Workload Pod templates.
    78  
    79       For example, for a [batch/v1.Job](https://kubernetes.io/docs/concepts/workloads/controllers/job/),
    80       Kueue adds the tolerations to the `.spec.template.spec.tolerations` field. This allows that the 
    81       workloads Pods to be scheduled on nodes having specific taints.
    82  
    83  ## ResourceFlavor taints
    84  
    85  To restrict the usage of a ResourceFlavor, you can configure the `.spec.nodeTaints` field.
    86  These taints should typically match the taints of the Nodes associated with the ResourceFlavor.
    87  
    88  Taints on the ResourceFlavor work similarly to [Node taints](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/).
    89  For Kueue to [admit](/docs/concepts#admission) a Workload to use the ResourceFlavor, the PodSpecs in the
    90  Workload should have a toleration for it. As opposed to the behavior for
    91  [ResourceFlavor labels](#resourceflavor-labels), Kueue does not add tolerations
    92  for the flavor taints.
    93  
    94  ## Empty ResourceFlavor
    95  
    96  If your cluster has homogeneous resources, or if you don't need to manage
    97  quotas for the different flavors of a resource separately, you can create a
    98  ResourceFlavor without any labels or taints. Such ResourceFlavor is called an
    99  empty ResourceFlavor and its definition looks like the following:
   100  
   101  ```yaml
   102  apiVersion: kueue.x-k8s.io/v1beta1
   103  kind: ResourceFlavor
   104  metadata:
   105    name: default-flavor
   106  ```
   107  
   108  ## What's next?
   109  
   110  - Learn about [cluster queues](/docs/concepts/cluster_queue).
   111  - Read the [API reference](/docs/reference/kueue.v1beta1/#kueue-x-k8s-io-v1beta1-ResourceFlavor) for `ResourceFlavor`