k8s.io/kubernetes@v1.29.3/test/e2e/dra/test-driver/README.md (about) 1 # dra-test-driver 2 3 This driver implements the controller and a resource kubelet plugin for dynamic 4 resource allocation. This is done in a single binary to minimize the amount of 5 boilerplate code. "Real" drivers could also implement both in different 6 binaries. 7 8 ## Usage 9 10 The driver could get deployed as a Deployment for the controller, with leader 11 election. A DaemonSet could get used for the kubelet plugin. The controller can 12 also run as a Kubernetes client outside of a cluster. The same works for the 13 kubelet plugin when using port forwarding. This is how it is used during 14 testing. 15 16 Valid parameters are key/value string pairs stored in a ConfigMap. 17 Those get copied into the ResourceClaimStatus with "user_" and "admin_" as 18 prefix, depending on whether they came from the ResourceClaim or ResourceClass. 19 They get stored in the `ResourceHandle` field as JSON map by the controller. 20 The kubelet plugin then sets these attributes as environment variables in each 21 container that uses the resource. 22 23 Resource availability is configurable and can simulate different scenarios: 24 25 - Network-attached resources, available on all nodes where the node driver runs, or 26 host-local resources, available only on the node whether they were allocated. 27 - Shared or unshared allocations. 28 - Unlimited or limited resources. The limit is a simple number of allocations 29 per cluster or node. 30 31 While the functionality itself is very limited, the code strives to showcase 32 best practices and supports metrics, leader election, and the same logging 33 options as Kubernetes. 34 35 ## Design 36 37 The binary itself is a Cobra command with two operations, `controller` and 38 `kubelet-plugin`. Logging is done with [contextual 39 logging](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/3077-contextual-logging). 40 41 The `k8s.io/dynamic-resource-allocation/controller` package implements the 42 interaction with ResourceClaims. It is generic and relies on an interface to 43 implement the actual driver logic. Long-term that part could be split out into 44 a reusable utility package. 45 46 The `k8s.io/dynamic-resource-allocation/kubelet-plugin` package implements the 47 interaction with kubelet, again relying only on the interface defined for the 48 kubelet<->dynamic resource allocation plugin interaction. 49 50 `app` is the driver itself with a very simple implementation of the interfaces. 51 52 ## Deployment 53 54 ### `local-up-cluster.sh` 55 56 To try out the feature, build Kubernetes, then in one console run: 57 ```console 58 RUNTIME_CONFIG="resource.k8s.io/v1alpha2" FEATURE_GATES=DynamicResourceAllocation=true ALLOW_PRIVILEGED=1 ./hack/local-up-cluster.sh -O 59 ``` 60 61 In another: 62 ```console 63 go run ./test/e2e/dra/test-driver --feature-gates ContextualLogging=true -v=5 controller 64 ``` 65 66 In yet another: 67 ```console 68 sudo mkdir -p /var/run/cdi && sudo chmod a+rwx /var/run/cdi /var/lib/kubelet/plugins_registry 69 go run ./test/e2e/dra/test-driver --feature-gates ContextualLogging=true -v=5 kubelet-plugin 70 ``` 71 72 And finally: 73 ```console 74 $ kubectl create -f test/e2e/dra/test-driver/deploy/example/resourceclass.yaml 75 resourceclass/example created 76 $ kubectl create -f test/e2e/dra/test-driver/deploy/example/pod-inline.yaml 77 configmap/pause-claim-parameters created 78 pod/pause created 79 80 $ kubectl get resourceclaims 81 NAME CLASSNAME ALLOCATIONMODE STATE AGE 82 pause-resource example WaitForFirstConsumer allocated,reserved 19s 83 84 $ kubectl get pods 85 NAME READY STATUS RESTARTS AGE 86 pause 1/1 Running 0 23s 87 ``` 88 89 There are also examples for other scenarios (multiple pods, multiple claims). 90 91 ### multi-node cluster 92 93 At this point there are no container images that contain the test driver and 94 therefore it cannot be deployed on "normal" clusters. 95 96 ## Prior art 97 98 Some of this code was derived from the 99 [external-resizer](https://github.com/kubernetes-csi/external-resizer/). `controller` 100 corresponds to the [controller 101 logic](https://github.com/kubernetes-csi/external-resizer/blob/master/pkg/controller/controller.go), 102 which in turn is similar to the 103 [sig-storage-lib-external-provisioner](https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner).