volcano.sh/volcano@v1.9.0/docs/user-guide/how_to_use_capacity_plugin.md (about) 1 # Capacity Plugin User Guide 2 3 ## Introduction 4 5 Capacity plugin is a replacement of proportion plugin, but instead of dividing the queue's deserved resources by weight, it realizes elastic queue capacity management i.e., queue's resource borrowing and lending mechanism by specifying the amount of deserved resources for each dimension resource of the queue. 6 7 A queue can use the idle resources of other queues, and when other queues submit jobs, they can reclaim the resources that have been lent, and the amount of reclaimed resources is the amount of queue's deserved resources. For more detail, please see [Capacity scheduling design](../design/capacity-scheduling.md) 8 9 ## Environment setup 10 11 ### Install volcano 12 13 Refer to [Install Guide](https://github.com/volcano-sh/volcano/blob/master/installer/README.md) to install volcano. 14 15 After installed, update the scheduler configuration: 16 17 ```shell 18 kubectl edit cm -n volcano-system volcano-scheduler-configmap 19 ``` 20 21 Make sure capacity plugin are enabled and remove proportion plugin. 22 23 Note: capacity and proportion plugin are in conflict, the two plugins cannot be used together. 24 25 ```yaml 26 kind: ConfigMap 27 apiVersion: v1 28 metadata: 29 name: volcano-scheduler-configmap 30 namespace: volcano-system 31 data: 32 volcano-scheduler.conf: | 33 actions: "enqueue, allocate, backfill" 34 tiers: 35 - plugins: 36 - name: priority 37 - name: gang 38 enablePreemptable: false 39 - name: conformance 40 - plugins: 41 - name: drf 42 enablePreemptable: false 43 - name: predicates 44 - name: capacity # add this field and remove proportion plugin. 45 - name: nodeorder 46 - name: binpack 47 ``` 48 49 ## Config queue's deserved resources 50 51 Assume there are two nodes and two queues named queue1 and queue2 in your kubernetes cluster, and each node has 4 CPU and 16Gi memory, then there will be total 8 CPU and 32Gi memory in your cluster. 52 53 ```yaml 54 allocatable: 55 cpu: "4" 56 memory: 16Gi 57 pods: "110" 58 ``` 59 60 config queue1's deserved field with 2 cpu and 8Gi memory. 61 62 ```yaml 63 apiVersion: scheduling.volcano.sh/v1beta1 64 kind: Queue 65 metadata: 66 name: queue1 67 spec: 68 reclaimable: true 69 deserved: # set the deserved field. 70 cpu: 2 71 memeory: 8Gi 72 ``` 73 74 config queue2's deserved field with 6 cpu and 24Gi memory. 75 76 ```yaml 77 apiVersion: scheduling.volcano.sh/v1beta1 78 kind: Queue 79 metadata: 80 name: queue2 81 spec: 82 reclaimable: true 83 deserved: # set the deserved field. 84 cpu: 6 85 memory: 24Gi 86 ``` 87 88 ## Submit pods to each queue 89 90 First, submit a deployment named demo-1 to queue1 with replicas=8 and each pod requests 1 cpu and 4Gi memory, because queue2 is idle, so queue1 can use the whole clusters' resources, and you can see that 8 pods are in Running state. 91 92 ```yaml 93 apiVersion: apps/v1 94 kind: Deployment 95 metadata: 96 name: demo-1 97 spec: 98 selector: 99 matchLabels: 100 app: demo-1 101 replicas: 8 102 template: 103 metadata: 104 labels: 105 app: demo-1 106 annotations: 107 scheduling.volcano.sh/queue-name: "queue1" # set the queue 108 spec: 109 schedulerName: volcano 110 containers: 111 - name: nginx 112 image: nginx:1.14.2 113 resources: 114 requests: 115 cpu: 1 116 memory: 4Gi 117 ports: 118 - containerPort: 80 119 ``` 120 121 Expected result: 122 123 ```shell 124 $ kubectl get po 125 NAME READY STATUS RESTARTS AGE 126 demo-1-7bc649f544-2wjg7 1/1 Running 0 5s 127 demo-1-7bc649f544-cvsmr 1/1 Running 0 5s 128 demo-1-7bc649f544-j5lzp 1/1 Running 0 5s 129 demo-1-7bc649f544-jvlbx 1/1 Running 0 5s 130 demo-1-7bc649f544-mzgg2 1/1 Running 0 5s 131 demo-1-7bc649f544-ntrs2 1/1 Running 0 5s 132 demo-1-7bc649f544-nv424 1/1 Running 0 5s 133 demo-1-7bc649f544-zd6d9 1/1 Running 0 5s 134 ``` 135 136 Then submit a deployment named demo-2 to queue2 with replicas=8 and each pod requests 1 cpu and 4Gi memory. 137 138 ```yaml 139 apiVersion: apps/v1 140 kind: Deployment 141 metadata: 142 name: demo-2 143 spec: 144 selector: 145 matchLabels: 146 app: demo-2 147 replicas: 8 148 template: 149 metadata: 150 labels: 151 app: demo-2 152 annotations: 153 scheduling.volcano.sh/queue-name: "queue2" # set the queue 154 spec: 155 schedulerName: volcano 156 containers: 157 - name: nginx 158 image: nginx:1.14.2 159 resources: 160 requests: 161 cpu: 1 162 memory: 4Gi 163 ports: 164 - containerPort: 80 165 ``` 166 167 Because queue1 occupied queue2's resources, so queue2 will reclaim its deserved resources with 6 cpu and 24Gi memory. And each pod of demo-2 request 1 cpu and 4Gi memory, so there will be 6 Pods in Running state of demo-2, and demo-1's pods will be evicted. 168 169 Finally, you can see that there are 2 Running pods in demo-1(belongs to queue1), and 6 Running pods in demo-2(belongs to queue2), which meets queue's deserved resources respectively. 170 171 ```shell 172 $ kubectl get po 173 NAME READY STATUS RESTARTS AGE 174 demo-1-7bc649f544-4vvdv 0/1 Pending 0 37s 175 demo-1-7bc649f544-c6mds 0/1 Pending 0 37s 176 demo-1-7bc649f544-j5lzp 1/1 Running 0 14m 177 demo-1-7bc649f544-mzgg2 1/1 Running 0 14m 178 demo-1-7bc649f544-pqdgk 0/1 Pending 0 37s 179 demo-1-7bc649f544-tx6wp 0/1 Pending 0 37s 180 demo-1-7bc649f544-wmshq 0/1 Pending 0 37s 181 demo-1-7bc649f544-wrhrr 0/1 Pending 0 37s 182 demo-2-6dfb86c49b-2jvgm 0/1 Pending 0 37s 183 demo-2-6dfb86c49b-dnjzv 1/1 Running 0 37s 184 demo-2-6dfb86c49b-fzvmp 1/1 Running 0 37s 185 demo-2-6dfb86c49b-jlf69 1/1 Running 0 37s 186 demo-2-6dfb86c49b-k62f7 1/1 Running 0 37s 187 demo-2-6dfb86c49b-k9b9v 1/1 Running 0 37s 188 demo-2-6dfb86c49b-rpzvg 0/1 Pending 0 37s 189 demo-2-6dfb86c49b-zch7w 1/1 Running 0 37s 190 ``` 191