agones.dev/agones@v1.54.0/examples/allocation-endpoint/README.md (about)

     1  # Allocation Endpoint
     2  
     3  This is a sample solution to enable an Allocation Endpoint proxy in front of GKE Agones clusters.
     4  
     5  In this folder you will find:
     6  
     7  1. Terraform modules that create resources in GCP
     8  2. The Allocation Endpoint Proxy code that gets deployed to Cloud Run
     9  3. An [ESP container](https://cloud.google.com/endpoints/docs/grpc/get-started-cloud-run) that gets deployed as a sidecar alongside the `agones-allocator`
    10  4. A sample client code to send allocation requests to the proxy
    11  5. Dockerfile that builds the proxy and scripts to push the image to a docker repository
    12  6. Documentation on how to use the solution.
    13  
    14  Here is the architecture of GCP resources created:
    15  
    16  ![architecture](https://github.com/googleforgames/agones/blob/main/examples/allocation-endpoint/architecture.png?raw=true)
    17  
    18  
    19  ## GKE cluster
    20  First and foremost you need to create clusters and install Agones to experiment with this solution.
    21  The clusters can be in the same GCP project as your Allocation Endpoint proxy or they can be in a different project.
    22  
    23  ---
    24  **NOTE**
    25  
    26  The solution has not been tested with non-GKE clusters.
    27  
    28  ---
    29  
    30  When creating GKE Clusters, [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) needs to be enabled. Please follow [GKE cluster setup](https://agones.dev/site/docs/installation/creating-cluster/gke/) and include workload-pool, e.g:
    31  
    32  <pre>
    33  gcloud container clusters create [NAME] \
    34    ...
    35    <b>--workload-pool=[GKE-PROJECT-ID].svc.id.goog \ </b>
    36  </pre>
    37  
    38  Install Agones on your cluster. You need to disable mTLS because the `agones-allocator` container will be the backend for ESP container.
    39  
    40  ```
    41  helm upgrade my-release --install --namespace agones-system --create-namespace agones/agones \
    42    --set agones.allocator.disableMTLS=true \
    43    --set agones.allocator.disableTLS=true \
    44    --set agones.allocator.service.http.enabled=false
    45  ```
    46  
    47  After installing Agones, deploy [ESP](https://cloud.google.com/endpoints/docs/grpc/specify-esp-v2-startup-options) which is an envoy based proxy, deployed as a sidecar along side `agones-alloator` container. Run the following to patch the service deployment, change the service port to ESP and add annotation to `agones-allocator` service account to impersonate GCP service account. 
    48  
    49  Replace [GKE-PROJECT-ID] in `patch-agones-allocator.yaml` with your project ID before running the scripts.
    50  
    51  ```
    52  kubectl patch deployment agones-allocator -n agones-system --patch-file patch-agones-allocator.yaml
    53  kubectl patch svc agones-allocator -n agones-system --type merge -p '{"spec":{"ports": [{"port": 443,"name":"https","targetPort":9443}]}}'
    54  kubectl annotate sa -n agones-system agones-allocator iam.gke.io/gcp-service-account=ae-esp-sa@[PROJECT-ID].iam.gserviceaccount.com
    55  ```
    56  
    57  ## Terraform 
    58  The terraform modules create resources in GCP:
    59  
    60  ```
    61  terraform apply \
    62    -var "project_id=[PROJECT-ID]" \
    63    -var "authorized_members=[\"serviceAccount:[SERVICE-ACCOUNT-EMAIL]\"]" \
    64    -var "clusters_info=[CLUSTERS-INFO]" \
    65    -var "workload-pool=[GKE-PROJECT-ID].svc.id.goog"
    66  ```
    67  
    68  `[CLUSTERS-INFO]` is in the form of `[{\"name\":\"cluster1\",\"endpoint\":\"34.83.14.82\",\"namespace\":\"default\",\"allocation_weight\":100},{...}]` deserializing to []ClusterInfo, defined in the `server/clusterselector.go`.
    69  
    70  - The `name` is a unique randomly selected name for the cluster.
    71  - The `endpoint` is the `agones-allocator` external IP.
    72  - The `namespace` is the game server namespace.
    73  - The `allocation_weight` is a value between 0 and 100, which sets the relative allocation rate a cluster receives compared to other clusters. By setting weight to zero, a cluster stops receiving allocation requests.
    74  
    75  `[SERVICE-ACCOUNT-EMAIL]` is the service account to be granted access the Allocation Endpoint. You need to have [the service account created](https://cloud.google.com/iam/docs/creating-managing-service-accounts) before running terraform.
    76  
    77  ## Server
    78  
    79  The Allocation Endpoint proxy code is in `./server` folder. You can make changes and run the following to build and push the image to your own GCR repository:
    80  
    81  ```
    82  docker build --tag gcr.io/[PROJECT-ID]/allocation-endpoint-proxy:[VERSION] .
    83  docker push gcr.io/[PROJECT-ID]/allocation-endpoint-proxy:[VERSION]
    84  ```
    85  
    86  If you are building your own image, you can set `ae_proxy_image` terraform variable to your image.
    87  
    88  ## Client
    89  
    90  The Allocation Endpoint client code is in `./client` folder. Get the Service Account Key for one of the Service Accounts in the list of `authorized_members` and put it under `sa_key.json`. Alternatively, you can leverage [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) to retrieve the access token using default service account from metadata serer, when deploying your client in a GCP solution, e.g. GKE.
    91  
    92  ```
    93  go run *.go --url=[CLOUD-RUN-ENDPOINT]
    94  
    95  ```
    96  
    97  `[CLOUD-RUN-ENDPOINT]` is the cloud run endpoint FQDN printed out after running the terraform. Leave out the scheme when setting the value e.g. `allocation-endpoint-proxy-<code>.a.run.app`.
    98  
    99  ## Future considerations
   100  - Requests using this example goes to public IP. For clusters in the same project you can instead leverage VPC with private IPs and remove dependency to the Service Account and Secret Manager to issue JWT in the proxy.
   101  - The proxy is skipping server cert validation. When you create a valid TLS cert, remove `InsecureSkipVerify: true``.
   102  - The solution should be compatible with non-GKE Agones clusters, but has not been tested.
   103