github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/deploy-manage/deploy/amazon_web_services/aws-deploy-kubernetes-kops.md (about)

     1  # Deploy Kubernetes with `kops`
     2  
     3  `kops` is one of the most popular open-source tools
     4  that enable you to deploy, manage, and upgrade a
     5  Kubernetes cluster in the cloud. By using `kops` you can
     6  quickly spin-up a highly-available Kubernetes cluster in
     7  a supported cloud platform.
     8  
     9  ## Prerequisites
    10  
    11  Before you can deploy Pachyderm on Amazon AWS with
    12  `kops`, you must have the following components configured:
    13  
    14  - Install [AWS CLI](https://aws.amazon.com/cli/)
    15  - Install [kubectl](https://kubernetes.io/docs/user-guide/prereqs/)
    16  - Install [kops](https://github.com/kubernetes/kops/blob/master/docs/install.md)
    17  - Install [pachctl](../../../../getting_started/local_installation/#install-pachctl)
    18  - Install [jq](https://stedolan.github.io/jq/download/)
    19  - Install [uuid](http://man7.org/linux/man-pages/man1/uuidgen.1.html)
    20  
    21  ## Configure `kops`
    22  
    23  [`kops`](https://github.com/kubernetes/kops/blob/master/docs/index.md), which stands for
    24  *Kubernetes Operations*, is an open-source tool that deploys
    25  a production-grade Kubernetes cluster on a cloud environment of choice.
    26  You need to have access to the
    27  AWS Management console to add an Identity and Access Management (IAM) user
    28  for `kops`.
    29  
    30  For more information about `kops`, see
    31  [kops AWS documentation](https://github.com/kubernetes/kops/blob/master/docs/getting_started/aws.md).
    32  These instructions provide more details about configuring
    33  additional cluster parameters, such as enabling version control
    34  or encryption on your S3 bucket, and so on.
    35  
    36  To configure `kops`, complete the following steps:
    37  
    38  1. In the IAM console or by using the command line, create a `kops` group
    39  with the following permissions:
    40  
    41     * `AmazonEC2FullAccess`
    42     * `AmazonRoute53FullAccess`
    43     * `AmazonS3FullAccess`
    44     * `IAMFullAccess`
    45     * `AmazonVPCFullAccess`
    46  
    47  1. Add a user that will create a Kubernetes cluster to that group.
    48  1. In the list of users, select that user and navigate to the
    49  **Security credentials** tab.
    50  1. Create an access key and save the access and secret keys in a
    51  location on your computer.
    52  1. Configure an AWS CLI client:
    53  
    54     ```shell
    55     aws configure
    56     ```
    57  
    58  1. Use the access and secret keys to configure the AWSL client.
    59  
    60  1. Create an S3 bucket for your cluster:
    61  
    62     ```shell
    63     aws s3api create-bucket --bucket <name> --region <region>
    64     ```
    65  
    66     **Example:**
    67  
    68     ```shell
    69     aws s3api create-bucket --bucket test-pachyderm --region us-east-1
    70     ```
    71  
    72     **System Response:**
    73  
    74     ```shell
    75     {
    76          "Location": "/test-pachyderm"
    77     }
    78     ```
    79  
    80  1. Optionally, configure DNS as described in [Configure DNS](https://github.com/kubernetes/kops/blob/master/docs/getting_started/aws.md#configure-dns).
    81  In this example, a gossip-based cluster that ends with `k8s.local`
    82  is deployed.
    83  
    84  1. Export the name of your cluster and the S3 bucket for the Kubernetes
    85  cluster as variables.
    86  
    87     **Example:**
    88  
    89     ```shell
    90     export NAME=test-pachyderm.k8s.local
    91     export KOPS_STATE_STORE=s3://test-pachyderm
    92     ```
    93  
    94  1. Create the cluster configuration:
    95  
    96     ```shell
    97     kops create cluster --zones <region> ${NAME}
    98     ```
    99  
   100  1. Optionally, edit your cluster:
   101  
   102     ```shell
   103     kops edit cluster ${NAME}
   104     ```
   105  
   106  1. Build and deploy the cluster:
   107  
   108     ```shell
   109     kops update cluster ${NAME} --yes
   110     ```
   111  
   112     The deployment might take some time.
   113  
   114  1. Run `kops cluster validate` periodically to monitor cluster deployment.
   115     When `kops` finishes deploying the cluster, you should see the output
   116     similar to the following:
   117  
   118     ```shell
   119     kops validate cluster
   120     ```
   121  
   122     **System Response:**
   123  
   124     ```shell
   125     Using cluster from kubectl context: test-pachyderm.k8s.local
   126  
   127     Validating cluster svetkars.k8s.local
   128  
   129     INSTANCE          GROUPS
   130     NAME              ROLE     MACHINETYPE MIN MAX SUBNETS
   131     master-us-west-2a Master   m3.medium   1   1   us-west-2a
   132     nodes             Node     t2.medium   2   2   us-west-2a
   133  
   134     NODE                                                   STATUS
   135     NAME                                           ROLE    READY
   136     ip-172-20-45-231.us-west-2.compute.internal    node    True
   137     ip-172-20-50-8.us-west-2.compute.internal      master  True
   138     ip-172-20-58-132.us-west-2.compute.internal    node    True
   139     ```
   140  
   141  1. Proceed to [Deploy Pachyderm on AWS](aws-deploy-pachyderm.md).