volcano.sh/volcano@v1.9.0/docs/design/custom-plugin.md (about)

     1  # Custom Plugin
     2  
     3  ## Background
     4  
     5  Until now, plugins like `binpack`, `drf`, `gang` are provided by official. But some users may want to implement the
     6  plugin by themselves. So if scheduler can dynamically load plugins, that will make it more flexible when handling
     7  different business scenarios.
     8  
     9  ## How to build a plugin
    10  
    11  ### 1. Build pluginable scheduler
    12  
    13  #### A. `alpine`
    14  
    15  If use `alpine` as base docker image, you should use `musl-gcc` to build scheduler.
    16  
    17  ```bash
    18  # install musl
    19  wget http://musl.libc.org/releases/musl-1.2.1.tar.gz
    20  tar -xf musl-1.2.1.tar.gz && cd musl-1.2.1
    21  ./configure
    22  make && sudo make install
    23  
    24  # build scheduler
    25  make images CC=/usr/local/musl/bin/musl-gcc SUPPORT_PLUGINS=yes
    26  ```
    27  
    28  If use `ubuntu` as base docker image
    29  
    30  ```bash
    31  make images SUPPORT_PLUGINS=yes
    32  ```
    33  
    34  ### 2. Coding
    35  
    36  ```go
    37  // magic.go
    38  
    39  package main // note!!! package must be named main
    40  
    41  import (
    42  	"volcano.sh/volcano/pkg/scheduler/framework"
    43  )
    44  
    45  const PluginName = "magic"
    46  
    47  type magicPlugin struct {}
    48  
    49  func (mp *magicPlugin) Name() string {
    50      return PluginName
    51  }
    52  
    53  func New(arguments framework.Arguments) framework.Plugin {  // `New` is PluginBuilder
    54  	return &magicPlugin{}
    55  }
    56  
    57  func (mp *magicPlugin) OnSessionOpen(ssn *framework.Session) {}
    58  
    59  func (mp *magicPlugin) OnSessionClose(ssn *framework.Session) {}
    60  ```
    61  
    62  ### 3. Build the plugin to .so
    63  
    64  #### A. Use musl-libc build plugin
    65  
    66  Because the default `vc-scheduler` base image is `alpine`, which only has `musl-libc`, so we should use `musl-gcc` to
    67  build the plugin.
    68  
    69  ```bash
    70  docker run -v `pwd`:/work golang:1.14-alpine sh -c "cd /work && apk add musl-dev gcc && go build -buildmode=plugin magic.go"
    71  ```
    72  
    73  Or build the plugin in local.
    74  
    75  ```bash
    76  # install musl
    77  wget http://musl.libc.org/releases/musl-1.2.1.tar.gz
    78  tar -xf musl-1.2.1.tar.gz && cd musl-1.2.1
    79  ./configure
    80  make && sudo make install
    81  
    82  # build plugin
    83  CC=/usr/local/musl/bin/musl-gcc CGO_ENABLED=1 go build -o plugins/magic.so -buildmode=plugin magic.go
    84  ```
    85  
    86  #### B. Use gnu-libc build plugin
    87  
    88  If want to use `ubuntu` as base image, you can use `gnu-libc` to build the plugin. Since most Linux OS have `gnu-libc`,
    89  you can just build the plugin in local.
    90  
    91  ```bash
    92  # default CC is gcc
    93  CGO_ENABLED=1 go build -o plugins/magic.so -buildmode=plugin magic.go
    94  ```
    95  
    96  ### 4. Add plugins into container
    97  
    98  Your can build your docker image
    99  
   100  ```dockerfile
   101  #Dockerfile
   102  FROM volcanosh/vc-scheduler:latest
   103  
   104  COPY plugins plugins
   105  ```
   106  
   107  ```
   108  docker build -t volcanosh/vc-scheduler:magic-plugins .
   109  ```
   110  
   111  
   112  
   113  Or just use `pvc` to mount these plugins
   114  
   115  ### 4. Specify deployment
   116  ```yaml
   117  ...
   118      containers:
   119      - name: volcano-scheduler
   120        image: volcanosh/vc-scheduler:magic-plugins
   121        args:
   122         - --logtostderr
   123         - --scheduler-conf=/volcano.scheduler/volcano-scheduler.conf
   124         - --enable-healthz=true
   125         - --enable-metrics=true
   126         - -v=3
   127         - --plugins-dir=plugins  # specify plugins dir path
   128         - 2>&1
   129  ```
   130  
   131  ### 5. Update volcano-scheduler-configmap
   132  
   133  Add your custom plugin name in configmap
   134  
   135  ```
   136  kubectl edit cm volcano-scheduler-configmap -n volcano-system
   137  ```
   138  
   139  
   140  
   141  ```yaml
   142  apiVersion: v1
   143  kind: ConfigMap
   144  metadata:
   145    name: volcano-scheduler-configmap
   146    namespace: volcano-system
   147  data:
   148    volcano-scheduler.conf: |
   149      actions: "enqueue, allocate, backfill"
   150      tiers:
   151      - plugins:
   152        - name: priority
   153        - name: gang
   154        - name: conformance
   155        - name: magic       # activate your custom plugin
   156      - plugins:
   157        - name: drf
   158        - name: predicates
   159        - name: proportion
   160        - name: nodeorder
   161        - name: binpack
   162  ```
   163  
   164  ## Note
   165  
   166  1. Plugins should be rebuilt after volcano source code modified.
   167  2. Plugin package name must be **main**.