volcano.sh/volcano@v1.9.0/docs/design/dynamic-conf.md (about)

     1  ## 1 Introduction
     2  
     3  ### 1.1 Project Background
     4  
     5  ​	Volcano consists of three components: Scheduler, Webhook, and Controller. The Scheduler component schedules jobs by performing a series of actions and plugins to find the most suitable node for them. The Controller Manager manages the lifecycle of CRD (Custom Resource Definition) resources and is mainly composed of Queue ControllerManager, PodGroupControllerManager, and VCJob ControllerManager. The Admission component is responsible for validating CRD API resources.
     6  ​	Currently, the startup configuration information for Volcano components is static and cannot be updated at runtime. When  locate a problem of volcano, usually the existing log level is not enough and the problem cannot be located, however, after restart volcano-scheduler and increasing the log level, the problem context is lost, making it impossible to continue locating. Therefore, it is convenient to update the log level without restarting volcano to identify the problem.
     7  
     8  ### 1.2 dynamic configuration candidates
     9  
    10  ​	The startup configuration information for Volcano components is currently static, and we need to select appropriate configurations and make them dynamic. I will identify all the static parameters used when launching Volcano and choose the configurations that are necessary to be dynamically adjustable under specific scenarios.
    11  
    12  ```
    13  -v, --v: The level of verbosity for log levels.
    14  ```
    15  
    16  #### Possible actual use scenarios
    17  
    18  1. `-v `The inability to update the log level of Volcano's Scheduler at runtime poses a significant burden on administrators during operation and maintenance. It is crucial to add the capability of dynamic configuration hot updates to enhance the efficiency of problem identification in production environments.
    19  
    20  ### 1.3 Project scope
    21  
    22  ​	Only focus on the static parameters that start the Schedule component
    23  
    24  ## 2 Implementation Plan
    25  
    26  ​	The vc-scheduler pod has an internal socket service to monitor changes in the klog.scok file. Because the socket service in the pod cannot be directly accessed in the k8s Node, the klog.sock file needs to be mounted from the pod to the cluster. Use The curl command modifies the klog.sock file.
    27  
    28  ![](images/dynamic-conf.jpg)
    29  
    30  example curl command example
    31  
    32  ```
    33  sudo curl --unix-socket /tmp/socks/klog.sock "http://localhost/setlevel?level=5&duration=60s"
    34  ```
    35  
    36  ## 3、Why not change klog level throuth change configmap
    37  1、Modifying configmap may affect other plugins.If  Volcano deployed in a customer cluster,  the configmap cannot be modified easily.
    38  2、It'll be confusing that both options and configmap has same parameters,but have different value.
    39  
    40  In deployment.yaml and configmap have different klog level.
    41  
    42  deployment.yaml
    43  
    44  ```
    45          - name: volcano-scheduler
    46            image: volcanosh/vc-scheduler:latest
    47            args:
    48              - -v=3
    49  ```
    50  
    51  configmap
    52  
    53  ```
    54    volcano-scheduler.conf: |
    55      actions: "enqueue, allocate, backfill"
    56      configurations:
    57      - name: dynamicConf
    58        arguments:
    59          klogLevel: "5"
    60  ```