k8s.io/kubernetes@v1.29.3/test/e2e/dra/test-driver/README.md (about)

     1  # dra-test-driver
     2  
     3  This driver implements the controller and a resource kubelet plugin for dynamic
     4  resource allocation. This is done in a single binary to minimize the amount of
     5  boilerplate code. "Real" drivers could also implement both in different
     6  binaries.
     7  
     8  ## Usage
     9  
    10  The driver could get deployed as a Deployment for the controller, with leader
    11  election. A DaemonSet could get used for the kubelet plugin. The controller can
    12  also run as a Kubernetes client outside of a cluster. The same works for the
    13  kubelet plugin when using port forwarding. This is how it is used during
    14  testing.
    15  
    16  Valid parameters are key/value string pairs stored in a ConfigMap.
    17  Those get copied into the ResourceClaimStatus with "user_" and "admin_" as
    18  prefix, depending on whether they came from the ResourceClaim or ResourceClass.
    19  They get stored in the `ResourceHandle` field as JSON map by the controller.
    20  The kubelet plugin then sets these attributes as environment variables in each
    21  container that uses the resource.
    22  
    23  Resource availability is configurable and can simulate different scenarios:
    24  
    25  - Network-attached resources, available on all nodes where the node driver runs, or
    26    host-local resources, available only on the node whether they were allocated.
    27  - Shared or unshared allocations.
    28  - Unlimited or limited resources. The limit is a simple number of allocations
    29    per cluster or node.
    30  
    31  While the functionality itself is very limited, the code strives to showcase
    32  best practices and supports metrics, leader election, and the same logging
    33  options as Kubernetes.
    34  
    35  ## Design
    36  
    37  The binary itself is a Cobra command with two operations, `controller` and
    38  `kubelet-plugin`. Logging is done with [contextual
    39  logging](https://github.com/kubernetes/enhancements/tree/master/keps/sig-instrumentation/3077-contextual-logging).
    40  
    41  The `k8s.io/dynamic-resource-allocation/controller` package implements the
    42  interaction with ResourceClaims. It is generic and relies on an interface to
    43  implement the actual driver logic. Long-term that part could be split out into
    44  a reusable utility package.
    45  
    46  The `k8s.io/dynamic-resource-allocation/kubelet-plugin` package implements the
    47  interaction with kubelet, again relying only on the interface defined for the
    48  kubelet<->dynamic resource allocation plugin interaction.
    49  
    50  `app` is the driver itself with a very simple implementation of the interfaces.
    51  
    52  ## Deployment
    53  
    54  ### `local-up-cluster.sh`
    55  
    56  To try out the feature, build Kubernetes, then in one console run:
    57  ```console
    58  RUNTIME_CONFIG="resource.k8s.io/v1alpha2" FEATURE_GATES=DynamicResourceAllocation=true ALLOW_PRIVILEGED=1 ./hack/local-up-cluster.sh -O
    59  ```
    60  
    61  In another:
    62  ```console
    63  go run ./test/e2e/dra/test-driver --feature-gates ContextualLogging=true -v=5 controller
    64  ```
    65  
    66  In yet another:
    67  ```console
    68  sudo mkdir -p /var/run/cdi && sudo chmod a+rwx /var/run/cdi /var/lib/kubelet/plugins_registry
    69  go run ./test/e2e/dra/test-driver --feature-gates ContextualLogging=true -v=5 kubelet-plugin
    70  ```
    71  
    72  And finally:
    73  ```console
    74  $ kubectl create -f test/e2e/dra/test-driver/deploy/example/resourceclass.yaml
    75  resourceclass/example created
    76  $ kubectl create -f test/e2e/dra/test-driver/deploy/example/pod-inline.yaml
    77  configmap/pause-claim-parameters created
    78  pod/pause created
    79  
    80  $ kubectl get resourceclaims
    81  NAME             CLASSNAME   ALLOCATIONMODE         STATE                AGE
    82  pause-resource   example     WaitForFirstConsumer   allocated,reserved   19s
    83  
    84  $ kubectl get pods
    85  NAME    READY   STATUS    RESTARTS   AGE
    86  pause   1/1     Running   0          23s
    87  ```
    88  
    89  There are also examples for other scenarios (multiple pods, multiple claims).
    90  
    91  ### multi-node cluster
    92  
    93  At this point there are no container images that contain the test driver and
    94  therefore it cannot be deployed on "normal" clusters.
    95  
    96  ## Prior art
    97  
    98  Some of this code was derived from the
    99  [external-resizer](https://github.com/kubernetes-csi/external-resizer/). `controller`
   100  corresponds to the [controller
   101  logic](https://github.com/kubernetes-csi/external-resizer/blob/master/pkg/controller/controller.go),
   102  which in turn is similar to the
   103  [sig-storage-lib-external-provisioner](https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner).