github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.9.x/deploy-manage/deploy/amazon_web_services/aws-deploy-kubernetes-kops.md (about)

     1  # Deploy Kubernetes with `kops`
     2  
     3  `kops` is one of the most popular open-source tools
     4  that enable you to deploy, manage, and upgrade a
     5  Kubernetes cluster in the cloud. By using `kops` you can
     6  quickly spin-up a highly-available Kubernetes cluster in
     7  a supported cloud platform.
     8  
     9  ## Prerequisites
    10  
    11  Before you can deploy Pachyderm on Amazon AWS with
    12  `kops`, you must have the following components configured:
    13  
    14  - Install [AWS CLI](https://aws.amazon.com/cli/)
    15  - Install [kubectl](https://kubernetes.io/docs/user-guide/prereqs/)
    16  - Install [kops](https://github.com/kubernetes/kops/blob/master/docs/install.md)
    17  - Install [pachctl](../../../../getting_started/local_installation/#install-pachctl)
    18  - Install [jq](https://stedolan.github.io/jq/download/)
    19  - Install [uuid](http://man7.org/linux/man-pages/man1/uuidgen.1.html)
    20  
    21  ## Configure `kops`
    22  
    23  [`kops`](https://github.com/kubernetes/kops/), which stands for
    24  *Kubernetes Operations*, is an open-source tool that deploys
    25  a production-grade Kubernetes cluster on a cloud environment of choice.
    26  You need to have access to the
    27  AWS Management console to add an Identity and Access Management (IAM) user
    28  for `kops`.
    29  
    30  For more information about `kops`, see
    31  [kops AWS documentation](https://github.com/kubernetes/kops/blob/master/docs/aws.md).
    32  These instructions provide more details about configuring
    33  additional cluster parameters, such as enabling version control
    34  or encryption on your S3 bucket, and so on.
    35  
    36  To configure `kops`, complete the following steps:
    37  
    38  1. In the IAM console or by using the command line, create a `kops` group
    39  with the following permissions:
    40  
    41     * `AmazonEC2FullAccess`
    42     * `AmazonRoute53FullAccess`
    43     * `AmazonS3FullAccess`
    44     * `IAMFullAccess`
    45     * `AmazonVPCFullAccess`
    46  
    47  1. Add a user that will create a Kubernetes cluster to that group.
    48  1. In the list of users, select that user and navigate to the
    49  **Security credentials** tab.
    50  1. Create an access key and save the access and secret keys in a
    51  location on your computer.
    52  1. Configure an AWS CLI client:
    53  
    54     ```shell
    55     $ aws configure
    56     ```
    57  
    58  1. Use the access and secret keys to configure the AWSL client.
    59  
    60  1. Create an S3 bucket for your cluster:
    61  
    62     ```shell
    63     $ aws s3api create-bucket --bucket <name> --region <region>
    64     ```
    65  
    66     **Example:**
    67  
    68     ```shell
    69     $ aws s3api create-bucket --bucket test-pachyderm --region us-east-1
    70     {
    71          "Location": "/test-pachyderm"
    72     }
    73     ```
    74  
    75  1. Optionally, configure DNS as described in [Configure DNS](https://github.com/kubernetes/kops/blob/master/docs/aws.md#configure-dns).
    76  In this example, a gossip-based cluster that ends with `k8s.local`
    77  is deployed.
    78  
    79  1. Export the name of your cluster and the S3 bucket for the Kubernetes
    80  cluster as variables.
    81  
    82     **Example:**
    83  
    84     ```shell
    85     export NAME=test-pachyderm.k8s.local
    86     export KOPS_STATE_STORE=s3://test-pachyderm
    87     ```
    88  
    89  1. Create the cluster configuration:
    90  
    91     ```shell
    92     kops create cluster --zones <region> ${NAME}
    93     ```
    94  
    95  1. Optionally, edit your cluster:
    96  
    97     ```shell
    98     kops edit cluster ${NAME}
    99     ```
   100  
   101  1. Build and deploy the cluster:
   102  
   103     ```shell
   104     kops update cluster ${NAME} --yes
   105     ```
   106  
   107     The deployment might take some time.
   108  
   109  1. Run `kops cluster validate` periodically to monitor cluster deployment.
   110     When `kops` finishes deploying the cluster, you should see the output
   111     similar to the following:
   112  
   113     ```shell
   114     $ kops validate cluster
   115     Using cluster from kubectl context: test-pachyderm.k8s.local
   116  
   117     Validating cluster svetkars.k8s.local
   118  
   119     INSTANCE          GROUPS
   120     NAME              ROLE     MACHINETYPE MIN MAX SUBNETS
   121     master-us-west-2a Master   m3.medium   1   1   us-west-2a
   122     nodes             Node     t2.medium   2   2   us-west-2a
   123  
   124     NODE                                                   STATUS
   125     NAME                                           ROLE    READY
   126     ip-172-20-45-231.us-west-2.compute.internal    node    True
   127     ip-172-20-50-8.us-west-2.compute.internal      master  True
   128     ip-172-20-58-132.us-west-2.compute.internal    node    True
   129     ```
   130  
   131  1. Proceed to [Deploy Pachyderm on AWS](aws-deploy-pachyderm.md).