sigs.k8s.io/gateway-api@v1.0.0/geps/gep-1867.md (about)

     1  # GEP-1867: Per-Gateway Infrastructure
     2  
     3  * Status: Experimental
     4  * Issue: [#1867](https://github.com/kubernetes-sigs/gateway-api/issues/1867)
     5  
     6  ## Overview
     7  
     8  `Gateway`s represent a piece of infrastructure implemented by cloud load balancers, in-cluster deployments, or other mechanisms.
     9  These often need vendor-specific configuration outside the scope of existing APIs (e.g. "size" or "version" of the infrastructure to provision).
    10  
    11  Today `GatewayClass.spec.parametersRef` is available to attach arbitrary configuration to a `GatewayClass`.
    12  
    13  This GEP will explain why that is not sufficient to meet common use cases, and introduce a new field - `infrastructure` - to address these cases.
    14  
    15  Related discussions:
    16  * [Support cluster-local Gateways](https://github.com/kubernetes-sigs/gateway-api/discussions/1247)
    17  * [Scaling Gateway Resources](https://github.com/kubernetes-sigs/gateway-api/discussions/1355)
    18  * [Manual deployments](https://github.com/kubernetes-sigs/gateway-api/issues/1687)
    19  * [Merging Gateways](https://github.com/kubernetes-sigs/gateway-api/pull/1863)
    20  * [In Cluster Gateway Deployments](https://github.com/kubernetes-sigs/gateway-api/pull/1757)
    21  
    22  ## Goals
    23  
    24  * Provide the ability to configure arbitrary (implementation specific) attributes about a **specific Gateway**.
    25  * Provide the ability to configure a standardized set of attributes about a **specific Gateway**.
    26  
    27  ## Why not GatewayClass parameters?
    28  
    29  `GatewayClass.spec.parametersRef` is the existing mechanism to configure arbitrary fields on a Gateway.
    30  However, this introduces operational challenges when configuring Gateways.
    31  
    32  ### Scope
    33  
    34  As a `Gateway` manager (with RBAC permissions to a specific `Gateway`) I should be able to declaratively make changes to that `Gateway` without the need for access to cluster-scoped resources (`GatewayClass`) and without affecting other `Gateways` managed by the same `GatewayClass`.
    35  This has been previously discussed in [this issue](https://github.com/kubernetes-sigs/gateway-api/issues/567).
    36  
    37  As a cluster scoped resource, `GatewayClass` does not meet this requirement.
    38  This restricts customization use cases to either a few pre-provisioned classes by the admin, or running in an environment where the "Infrastructure Provider" and "Cluster Operator" are the same roles.
    39  The distinction between these roles is explicitly called out on the [homepage](https://gateway-api.sigs.k8s.io/#what-is-the-gateway-api).
    40  
    41  ### Custom Resource
    42  
    43  `parametersRef` is entirely a generic implementation-specific meaning.
    44  This means implementations will either need a custom CRD or use untyped resources like ConfigMap.
    45  Neither of these have any consistency between implementations.
    46  While there will always be some vendor-specific requirements, there are also a number of configuration aspects of a Gateway that are common between implementations.
    47  However, these cannot currently be expressed in a vendor-neutral way.
    48  
    49  The original motivation behind `parametersRef` was for implementation specific concepts, while portable comments could be added into the API as first-class fields, but this has not been done (yet).
    50  
    51  Additionally, there is hesitancy to use a CRD (which leads to CRD proliferation), which pushes users towards untyped ConfigMaps which are not much better than annotations.
    52  The scoping, as mentioned above, is also a bit awkward of a cluster scoped resource pointing to a namespaced object.
    53  
    54  ### Separation of concerns
    55  
    56  While there is value out of providing class-wide options as defaults, there is also value in providing these options on the object (Gateway) directly.
    57  
    58  Some parallels in existing APIs:
    59  
    60  [Policy Attachment](https://gateway-api.sigs.k8s.io/reference/policy-attachment) offers a hierarchy of defaults and overrides, allowing attachment to GatewayClass and Gateway.
    61  This is similar to our needs here, but representing infrastructure configuration as a "Policy" is a bit problematic, and the existing mechanisms have no hierarchy.
    62  
    63  In core Kubernetes, Pods declare their requirements (for example, CPU requests) inline in the Pod resource; there is not a `ResourceClass` API that abstracts these further.
    64  These higher level abstractions are handled by layered APIs (whether this is a CRD, an admission webhook, CI/CD tooling, etc).
    65  This allows users the flexibility to easily configure things per-pod basis.
    66  If the infrastructure admin wants to impose defaults or requirements on this flexibility, they are able to do so (in fact, `LimitRanger` provides a built in mechanism to do so).
    67  
    68  ### Dynamic Changes
    69  
    70  Currently, the spec recommends `GatewayClass` to be used as a *template*.
    71  Changes to it are not expected to change deployed `Gateway`s.
    72  
    73  This makes usage problematic in a declarative way.
    74  For example, if I wanted to represent a `version` field and change that to trigger an upgrade, I would need to create an entirely new `Gateway`.
    75  
    76  ## API
    77  
    78  In order to address the concerns above, I propose a standard `infrastructure` API is added to `Gateway` and `GatewayClass`.
    79  Note the important part of this is the `Gateway` change; the `GatewayClass` aspect is mostly for consistency.
    80  
    81  The exact fields are out of scope for this GEP and will be handled by additional GEPs.
    82  One example GEP already depending on this is [GEP-1651](/geps/gep-1651).
    83  
    84  The fields as defined below are, of course, not useful.
    85  This is intended as a basis for other PRs, not to provide value on its own.
    86  This GEP will remain in provisional until at least one field is ready to be promoted.
    87  
    88  ```go
    89  type GatewaySpec struct {
    90    // Infrastructure defines infrastructure level attributes about this Gateway instance.
    91    Infrastructure GatewayInfrastructure `json:"infrastructure"`
    92    // ...
    93  }
    94  
    95  type GatewayClassSpec struct {
    96    // Infrastructure defines infrastructure level attributes for all Gateways in this class.
    97    // A Gateway may provide configuration for the same values; as all fields in GatewayInfrastructure are implementation specific,
    98    // the merging logic between these is as well. However, the GatewayClass is generally expected to be providing defaults
    99    // rather than overrides.
   100    Infrastructure GatewayClassInfrastructure `json:"infrastructure"`
   101    // ...
   102  }
   103  
   104  type GatewayInfrastructure struct {
   105    // ParametersRef provides a arbitrary implementation-specific configuration for
   106    // fields not expressed directly in this struct.
   107    // This follows the same semantics as GatewayClass's ParametersRef, but lives on the Gateway.
   108    ParametersRef ParametersReference
   109  }
   110  
   111  type GatewayClassInfrastructure struct {
   112  }
   113  ```
   114  
   115  ### API Principles
   116  
   117  For any given field, we will need to make two decisions:
   118  * whether this should be a first-class field or a generic `parametersRef`.
   119  * whether this field should be configurable on a Gateway and/or GatewayClass level
   120  
   121  The choice to use an extension (`parametersRef`) or first-class field is a well known problem across the API, and the same logic will be used here.
   122  Fields that are generally portable across implementations and have wide-spread demand and use cases will be promoted to first-class fields,
   123  while vendor specific or niche fields will remain extensions.
   124  Because infrastructure is somewhat inherently implementation specific, it is likely most fields will be Extended or ImplementationSpecific.
   125  However, there are still a variety of concepts that have some meaning between implementations that can provide value to users.
   126  
   127  Introduction at Gateway or GatewayClass level will depend on the specific field and use cases for the field.
   128  In general, it makes sense to provide defaults (GatewayClass) and specific settings (Gateway) for most fields, but
   129  this will be evaluated on a case-by-case basis.
   130  
   131  ### Status
   132  
   133  The API should likely expose some status. However, it is not yet clear what that will look like.
   134  This will be addressed prior to promotion beyond "Provisional".