volcano.sh/volcano@v1.9.0/README.md (about)

     1  <a href="https://volcano.sh/">
     2      <img src="https://raw.githubusercontent.com/volcano-sh/volcano/master/docs/images/volcano-horizontal-color.png"/>
     3  </a>
     4  
     5  -------
     6  
     7  [![Build Status](https://travis-ci.org/volcano-sh/volcano.svg?branch=master)](https://travis-ci.org/volcano-sh/volcano)
     8  [![Go Report Card](https://goreportcard.com/badge/github.com/volcano-sh/volcano)](https://goreportcard.com/report/github.com/volcano-sh/volcano)
     9  [![RepoSize](https://img.shields.io/github/repo-size/volcano-sh/volcano.svg)](https://github.com/volcano-sh/volcano)
    10  [![Release](https://img.shields.io/github/release/volcano-sh/volcano.svg)](https://github.com/volcano-sh/volcano/releases)
    11  [![LICENSE](https://img.shields.io/github/license/volcano-sh/volcano.svg)](https://github.com/volcano-sh/volcano/blob/master/LICENSE)
    12  [![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/3012/badge)](https://bestpractices.coreinfrastructure.org/projects/3012)
    13  
    14  
    15  [Volcano](https://volcano.sh/) is a batch system built on Kubernetes. It provides a suite of mechanisms that are commonly required by
    16  many classes of batch & elastic workload including: machine learning/deep learning, bioinformatics/genomics and
    17  other "big data" applications. These types of applications typically run on generalized domain frameworks like
    18  TensorFlow, Spark, Ray, PyTorch, MPI, etc, which Volcano integrates with.
    19  
    20  Volcano builds upon a decade and a half of experience running a wide
    21  variety of high performance workloads at scale using several systems
    22  and platforms, combined with best-of-breed ideas and practices from
    23  the open source community.
    24  
    25  Until June 2021, Volcano has been widely used around the world at a variety of industries such as Internet/Cloud/Finance/
    26  Manufacturing/Medical. More than 20 companies or institutions are not only end users but also active contributors. Hundreds
    27  of contributors are taking active part in the code commit/PR review/issue discussion/docs update and design provision. We
    28  are looking forward to your participation.
    29  
    30  **NOTE**: the scheduler is built based on [kube-batch](https://github.com/kubernetes-sigs/kube-batch);
    31  refer to [#241](https://github.com/volcano-sh/volcano/issues/241) and [#288](https://github.com/volcano-sh/volcano/pull/288) for more detail.
    32  
    33  ![cncf_logo](docs/images/cncf-logo.png)
    34  
    35  Volcano is an incubating project of the [Cloud Native Computing Foundation](https://cncf.io/) (CNCF). Please consider joining the CNCF if you are an organization that wants to take an active role in supporting the growth and evolution of the cloud native ecosystem. 
    36  
    37  ## Overall Architecture
    38  
    39  ![volcano](docs/images/volcano-architecture.png)
    40  
    41  ## Talks
    42  
    43  - [Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU](https://sched.co/MPi7)
    44  - [Volcano 在 Kubernetes 中运行高性能作业实践 @ ArchSummit 2019](https://archsummit.infoq.cn/2019/shenzhen/presentation/1817)
    45  - [Volcano：基于云原生的高密计算解决方案 @ Huawei Connection 2019](https://e.huawei.com/cn/material/event/HC/09099dce0070415e9f26ada51b2216d7)
    46  - [Improving Performance of Deep Learning Workloads With Volcano @ KubeCon 2019 NA](https://sched.co/UaZi)
    47  - [Batch Capability of Kubernetes Intro @ KubeCon 2019 NA](https://sched.co/Uajv)
    48  - [Intro: Kubernetes Batch Scheduling @ KubeCon 2019 EU](https://sched.co/MPi7)
    49  
    50  
    51  ## Ecosystem
    52  
    53  - [spark-operator](https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/volcano-integration.md)
    54  - [kubeflow/tf-operator](https://www.kubeflow.org/docs/use-cases/job-scheduling/)
    55  - [kubeflow/arena](https://github.com/kubeflow/arena/blob/master/docs/training/volcanojob/volcanojob.md)
    56  - [Horovod/MPI](https://github.com/volcano-sh/volcano/tree/master/example/integrations/mpi)
    57  - [paddlepaddle](https://github.com/volcano-sh/volcano/tree/master/example/integrations/paddlepaddle)
    58  - [cromwell](https://github.com/broadinstitute/cromwell/blob/develop/docs/backends/Volcano.md)
    59  - [KubeRay](https://ray-project.github.io/kuberay/guidance/volcano-integration)
    60  
    61  ## Quick Start Guide
    62  
    63  ### Prerequisites
    64  
    65  - Kubernetes 1.12+ with CRD support
    66  
    67  
    68  You can try Volcano by one of the following two ways.
    69  
    70  Note: 
    71  * For Kubernetes v1.17+ use CRDs under config/crd/bases (recommended)
    72  * For Kubernetes versions < v1.16 use CRDs under config/crd/v1beta1 (deprecated)
    73  
    74  ### Install with YAML files
    75  
    76  Install Volcano on an existing Kubernetes cluster. This way is both available for x86_64 and arm64 architecture.
    77  
    78  ```
    79  kubectl apply -f https://raw.githubusercontent.com/volcano-sh/volcano/release-1.9/installer/volcano-development.yaml
    80  ```
    81  
    82  Enjoy! Volcano will create the following resources in `volcano-system` namespace.
    83  
    84  
    85  ```
    86  NAME                                       READY   STATUS      RESTARTS   AGE
    87  pod/volcano-admission-5bd5756f79-dnr4l     1/1     Running     0          96s
    88  pod/volcano-admission-init-4hjpx           0/1     Completed   0          96s
    89  pod/volcano-controllers-687948d9c8-nw4b4   1/1     Running     0          96s
    90  pod/volcano-scheduler-94998fc64-4z8kh      1/1     Running     0          96s
    91  
    92  NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
    93  service/volcano-admission-service   ClusterIP   10.98.152.108   <none>        443/TCP   96s
    94  
    95  NAME                                  READY   UP-TO-DATE   AVAILABLE   AGE
    96  deployment.apps/volcano-admission     1/1     1            1           96s
    97  deployment.apps/volcano-controllers   1/1     1            1           96s
    98  deployment.apps/volcano-scheduler     1/1     1            1           96s
    99  
   100  NAME                                             DESIRED   CURRENT   READY   AGE
   101  replicaset.apps/volcano-admission-5bd5756f79     1         1         1       96s
   102  replicaset.apps/volcano-controllers-687948d9c8   1         1         1       96s
   103  replicaset.apps/volcano-scheduler-94998fc64      1         1         1       96s
   104  
   105  NAME                               COMPLETIONS   DURATION   AGE
   106  job.batch/volcano-admission-init   1/1           48s        96s
   107  
   108  ```
   109  
   110  ### Install via helm
   111  
   112  To install official release, please visit to [helm-charts](https://github.com/volcano-sh/helm-charts) for details.
   113  
   114  ```bash
   115  helm repo add volcano-sh https://volcano-sh.github.io/helm-charts
   116  helm install volcano volcano-sh/volcano -n volcano-system --create-namespace
   117  ```
   118  
   119  Install from source code for developers:
   120  
   121  ```bash
   122  helm install volcano installer/helm/chart/volcano --namespace volcano-system --create-namespace
   123  
   124  # list helm release
   125  helm list -n volcano-system
   126  ```
   127  
   128  ### Install from code
   129  
   130  If you don't have a kubernetes cluster, try one-click install from code base:
   131  
   132  ```bash
   133  ./hack/local-up-volcano.sh
   134  ```
   135  
   136  This way is only available for x86_64 temporarily.
   137  
   138  ### Install monitoring system
   139  
   140  If you want to get prometheus and grafana volcano dashboard after volcano installed, try following commands:
   141  
   142  ```bash
   143  make TAG=v1.9.0 generate-yaml
   144  kubectl create -f _output/release/volcano-monitoring-v1.9.0.yaml
   145  ```
   146  
   147  ## Kubernetes compatibility
   148  
   149  |                       | Kubernetes 1.17 | Kubernetes 1.18 | Kubernetes 1.19 | Kubernetes 1.20 | Kubernetes 1.21 | Kubernetes 1.22 | Kubernetes 1.23 | Kubernetes 1.24 | Kubernetes 1.25 | Kubernetes 1.26 | Kubernetes 1.27 | Kubernetes 1.28 | Kubernetes 1.29 |
   150  |-----------------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|-----------------|
   151  | Volcano v1.6          | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | -               | -               | -               | -               | -               | -               |
   152  | Volcano v1.7          | -               | -               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | -               |
   153  | Volcano v1.8          | -               | -               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | -               |
   154  | Volcano v1.9          | -               | -               | -               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               |
   155  | Volcano HEAD (master) | -               | -               | -               | -               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               | ✓               |
   156  
   157  Key:
   158  * `✓` Volcano and the Kubernetes version are exactly compatible.
   159  * `+` Volcano has features or API objects that may not be present in the Kubernetes version.
   160  * `-` The Kubernetes version has features or API objects that Volcano can't use.
   161  
   162  
   163  ## Meeting
   164  
   165  Community weekly meeting for Asia: 15:00 - 16:00 (UTC+8) Friday. ([Convert to your timezone.](https://www.thetimezoneconverter.com/?t=10%3A00&tz=GMT%2B8&))
   166  
   167  Community biweekly meeting for America: 08:30 - 09:30 (UTC-8) Thursday. ([Convert to your timezone.](https://www.thetimezoneconverter.com/?t=10%3A00&tz=GMT%2B8&))
   168  
   169  Community meeting for Europe is ongoing on demand now. If you have some ideas or topics to discuss, please leave message
   170  in the [slack](https://cloud-native.slack.com/archives/C011GJDQS0N). Maintainers will contact with you and book an open meeting for that.
   171  
   172  Resources:
   173  - [Meeting notes and agenda](https://docs.google.com/document/d/1YLbF8zjZBiR9PbXQPB22iuc_L0Oui5A1lddVfRnZrqs/edit)
   174  - [Meeting link](https://zoom.us/j/91804791393)
   175  - [Meeting Calendar](https://calendar.google.com/calendar/b/1/embed?src=volcano.sh.bot@gmail.com) | [Subscribe](https://calendar.google.com/calendar/b/1?cid=dm9sY2Fuby5zaC5ib3RAZ21haWwuY29t)
   176  
   177  ## Contact
   178  
   179  If you have any question, feel free to reach out to us in the following ways:
   180  
   181  [Volcano Slack Channel](https://cloud-native.slack.com/archives/C011GJDQS0N) | [Join](https://slack.cncf.io/)
   182  
   183  [Mailing List](https://groups.google.com/forum/#!forum/volcano-sh)
   184  
   185  Wechat: Add WeChat account `k8s2222` (华为云小助手2号) to let her pull you into the group.