github.com/oam-dev/kubevela@v1.9.11/design/vela-core/environment.md (about)

     1  # Making Environments For Application Deployment
     2  
     3  ## Background
     4  
     5  An application development team usually needs to initialize some shared environments for developers to deploy applications to.
     6  For example, a team would initialize two environments, a *dev* environment for testing applications,
     7  and a *prod* environment for running applications to serve live traffic.
     8  
     9  The environment is a logical concept grouping common resources that many applications would depend on. Below are a list of cases:
    10  
    11  - Kubernetes clusters. These include not only existing clusters but also new clusters to create during environment setup.
    12  - Admin policies. Production environment would usually set global policies for all application deployments,
    13    like chaos testing, SLO requirements, security scanning, misconfiguration detection.
    14  - System components. That includes installing system level Operators & CRDS like FluxCD, KEDA, OpenKruise,
    15    shared Namespaces, Observability (Prometheus, Grafana, Loki), Terraform Controller, KubeFlow Controller, etc.
    16  - Shared services. Environments contains a variety of system services that are be shared by applications.
    17    These resources can include DB, cache, load balancers, API Gateways, etc.
    18  
    19  In this doc, we will provide solutions and case studies on how to use KubeVela to make environments, initialize shared resources,
    20  and compose application rollout across multiple environments.
    21  
    22  
    23  ## Using KubeVela to initialize Environment
    24  
    25  An environment is just a logical concept that groups a bundle of resources for multiple applications to share.
    26  To make an environment, we first need to initialize the shared resources.
    27  
    28  When initializing an environment, there is one additional requirement:
    29  
    30  - The ability to describe dependencies between environment modules. For example, both *dev* and *prod* environments
    31    would depend on a common module that installs operators both *dev* and *prod* would use.
    32  
    33  To setup shared resources and define dependencies, we propose to use Application with two Workflow Steps
    34  to achieve the requirements. An example looks like below:
    35  
    36  ```yaml
    37  apiVersion: core.oam.dev/v1beta1
    38  kind: Application
    39  metadata:
    40    name: setup-dev-env
    41  spec:
    42    # Using Application to deploy resources that makes up an environment
    43  
    44    components:
    45    - name: <env-component-name>
    46      type: <env-component-type>
    47      properties:
    48        <env-component-props>
    49  
    50    policies:
    51    - name: <env-policy-name>
    52      type: <env-policy-type>
    53      properties:
    54        <env-policy-props>
    55  
    56    workflow:
    57    - name: wait-dependencies
    58      # depends-on-app step will wait until the dependent app's status turns to Running phase
    59      type: depends-on-app
    60      properties:
    61        name: <application name>
    62        namespace: <application namespace>
    63        
    64    - name: apply-self
    65      # apply-application will apply all components of this Application per se
    66      type: apply-application
    67  ```
    68  
    69  
    70  ## Using KubeVela to compose multi-env application rollout
    71  
    72  Once we have initialized an environment, we can then deploy applications across environments.
    73  For example, we can deploy an application to dev env first, then verify the app is working, and promote to prod env at the end.
    74  
    75  We can use the following Definition to achieve that:
    76  
    77  - env-binding Policy: This defines the config patch and placement strategy per env.
    78  - deploy2env WorkflowStep: This picks which policy and env to deploy the app to.
    79  - suspend WorkflowStep: This will pause the workflow for some manual validation.
    80  
    81  Below is an example:
    82  
    83  ```yaml
    84  apiVersion: core.oam.dev/v1beta1
    85  kind: Application
    86  metadata:
    87    name: multi-env-demo
    88  spec:
    89    components:
    90      - name: myimage-server
    91        type: webservice
    92        properties:
    93          image: myimage:v1.1
    94          port: 80
    95  
    96    policies:
    97      - name: my-binding-policy
    98        type: env-binding
    99        properties:
   100          envs:
   101            - name: test
   102  
   103              patch: # overlay patch on above components
   104                components:
   105                  - name: myimage-server
   106                    type: webservice
   107                    properties:
   108                      image: myimage:v1.2
   109                      port: 80
   110  
   111              placement: # selecting the cluster to deploy to
   112                clusterSelector:
   113                  labels:
   114                    purpose: test
   115  
   116            - name: prod
   117              placement:
   118                clusterSelector:
   119                  labels:
   120                    purpose: prod
   121  
   122    workflow:
   123      steps:
   124        - name: deploy-test-env
   125          type: deploy2env
   126          properties:
   127            policy: my-binding-policy
   128            env: test
   129  
   130        - name: manual-approval 
   131          type: suspend
   132  
   133        - name: deploy-prod-env
   134          type: deploy2env
   135          properties:
   136            policy: my-binding-policy
   137            env: prod
   138  ```
   139  
   140  Here're more details for above example:
   141  
   142  - It sets up an `env-binding` policy which defines two envs for users to use.
   143    In each env, it defines the config patch and placement strategy specific to this env.
   144  - When the application runs, it triggers the following workflow:
   145    - First it picks the policy, and picks the `test` env which is also defined inside the policy.
   146    - Then the `deploy2env` step will load the policy data, picks the `test` env specific config section.
   147      This step will render the final Application with patch data,
   148      and picks the cluster to deploy to based on given placement strategy,
   149      and finally deploys the Application to the cluster.
   150    - Then it runs `suspend` step, which acts as an approval gate until user validation 
   151    - Finally, it runs `deploy2env` step again. Only this time it picks the `prod` env.
   152      This step will render the final Application with patch data,
   153      and picks the cluster to deploy to based on given placement strategy,
   154      and finally deploys the Application to the cluster.
   155  
   156  
   157  ## Implementation Plan
   158  
   159  We will discuss the details of implementation plan in this section.
   160  
   161  ## Considerations
   162  
   163  
   164  ## Appendix
   165  
   166  ### Creating Kubernetes cluster
   167  
   168  In this section, we will show how to create a Kubernetes cluster on Alibaba Cloud using Terraform.
   169  
   170  We will first setup Terraform Alibab provider:
   171  
   172  ```yaml
   173  apiVersion: core.oam.dev/v1beta1
   174  kind: Application
   175  metadata:
   176    name: terraform-alibaba
   177    namespace: vela-system
   178  spec:
   179    components:
   180      - name: default
   181        type: raw
   182        properties:
   183          apiVersion: terraform.core.oam.dev/v1beta1
   184          kind: Provider
   185          metadata:
   186            namespace: default
   187          spec:
   188            provider: alibaba
   189            region: cn-hongkong
   190            credentials:
   191              source: Secret
   192              secretRef:
   193                namespace: vela-system
   194                name: alibaba-account-creds
   195                key: credentials
   196    workflow:
   197    - name: wait-dependencies
   198      type: depends-on-app
   199      properties:
   200        name: terraform
   201        namespace: vela-system
   202        
   203    - name: apply-self
   204      type: apply-application
   205  ```
   206  
   207  Then we will create a Kubernetes cluster:
   208  
   209  ```yaml
   210  apiVersion: core.oam.dev/v1beta1
   211  kind: Application
   212  metadata:
   213    name: managed-cluster
   214    namespace: vela-system
   215  spec:
   216    components:
   217      - name: ack-worker
   218        type: alibaba-ack
   219        properties:
   220          writeConnectionSecretToRef:
   221            name: ack-conn
   222            namespace: vela-system
   223    workflow:
   224      steps:
   225        - name: wait-dependencies
   226          type: depends-on-app
   227          properties:
   228            name: terraform-alibaba
   229            namespace: vela-system
   230  
   231        - name: wait-dependencies
   232          type: depends-on-app
   233          properties:
   234            name: ocm-cluster-manager
   235            namespace: vela-system
   236            
   237        - name: terraform-ack
   238          type: create-ack
   239          properties:
   240            component: ack-worker
   241          outputs:
   242            - name: connInfo
   243              valueFrom: connInfo
   244  
   245        - name: register-ack
   246          type: register-cluster
   247          inputs:
   248            - from: connInfo
   249              parameterKey: connInfo
   250          properties:
   251            # user should set public network address of APIServer
   252            hubAPIServer: {{ public network address of APIServer }}
   253            env: prod
   254            initNameSpace: default
   255            patchLabels:
   256              purpose: test
   257  ```
   258  
   259  It will wait for dependent system components to be installed first, and then create the cluster,
   260  and finally register the cluster in K8s API.