github.com/projectcontour/contour@v1.28.2/site/content/docs/v1.0.1/httpproxy.md (about) 1 The [Ingress][1] object was added to Kubernetes in version 1.1 to describe properties of a cluster-wide reverse HTTP proxy. 2 Since that time, the Ingress object has not progressed beyond the beta stage, and its stagnation inspired an [explosion of annotations][2] to express missing properties of HTTP routing. 3 4 The goal of the `HTTPProxy` (previously `IngressRoute`) Custom Resource Definition (CRD) is to expand upon the functionality of the Ingress API to allow for a richer user experience as well addressing the limitations of the latter's use in multi tenent environments. 5 6 ## Key HTTPProxy Benefits 7 8 - Safely supports multi-team Kubernetes clusters, with the ability to limit which Namespaces may configure virtual hosts and TLS credentials. 9 - Enables including of routing configuration for a path or domain from another HTTPProxy, possibly in another Namespace. 10 - Accepts multiple services within a single route and load balances traffic across them. 11 - Natively allows defining service weighting and load balancing strategy without annotations. 12 - Validation of HTTPProxy objects at creation time and status reporting for post-creation validity. 13 14 ## Ingress to HTTPProxy 15 16 A minimal Ingress object might look like: 17 18 ```yaml 19 # ingress.yaml 20 apiVersion: networking.k8s.io/v1beta1 21 kind: Ingress 22 metadata: 23 name: basic 24 spec: 25 rules: 26 - host: foo-basic.bar.com 27 http: 28 paths: 29 - backend: 30 serviceName: s1 31 servicePort: 80 32 ``` 33 34 This Ingress object, named `basic`, will route incoming HTTP traffic with a `Host:` header for `foo-basic.bar.com` to a Service named `s1` on port `80`. 35 Implementing similar behavior using an HTTPProxy looks like this: 36 37 {% highlight yaml linenos %} 38 # httpproxy.yaml 39 apiVersion: projectcontour.io/v1 40 kind: HTTPProxy 41 metadata: 42 name: basic 43 spec: 44 virtualhost: 45 fqdn: foo-basic.bar.com 46 routes: 47 - conditions: 48 - prefix: / 49 services: 50 - name: s1 51 port: 80 52 {% endhighlight %} 53 54 **Lines 1-5**: As with all other Kubernetes objects, an HTTPProxy needs apiVersion, kind, and metadata fields. 55 56 **Lines 7-8**: The presence of the `virtualhost` field indicates that this is a root HTTPProxy that is the top level entry point for this domain. 57 The `fqdn` field specifies the fully qualified domain name that will be used to match against `Host:` HTTP headers. 58 59 **Lines 9-14**: Each HTTPProxy **must** have one or more routes, each of which **must** have one or more services which will handle the HTTP traffic. In addition, each route **may** have one or more conditions to match against. 60 61 **Lines 12-14**: The `services` field is an array of named Service & Port combinations that will be used for this HTTPProxy path. 62 HTTP traffic will be sent directly to the Endpoints corresponding to the Service. 63 64 ## Interacting with HTTPProxies 65 66 As with all Kubernetes objects, you can use `kubectl` to create, list, describe, edit, and delete HTTPProxy CRDs. 67 68 Creating an HTTPProxy: 69 70 ```bash 71 $ kubectl create -f basic.httpproxy.yaml 72 httpproxy "basic" created 73 ``` 74 75 Listing HTTPProxies: 76 77 ```bash 78 $ kubectl get httpproxy 79 NAME AGE 80 basic 24s 81 ``` 82 83 Describing HTTPProxy: 84 85 ```bash 86 $ kubectl describe httpproxy basic 87 Name: basic 88 Namespace: default 89 Labels: <none> 90 API Version: projectcontour.io/v1 91 Kind: HTTPProxy 92 Metadata: 93 Cluster Name: 94 Creation Timestamp: 2019-07-05T19:26:54Z 95 Resource Version: 19373717 96 Self Link: /apis/projectcontour.io/v1/namespaces/default/httpproxy/basic 97 UID: 6036a9d7-8089-11e8-ab00-f80f4182762e 98 Spec: 99 Routes: 100 Conditions: 101 Prefix: / 102 Services: 103 Name: s1 104 Port: 80 105 Virtualhost: 106 Fqdn: foo-basic.bar.com 107 Events: <none> 108 ``` 109 110 Deleting HTTPProxies: 111 112 ```bash 113 $ kubectl delete httpproxy basic 114 httpproxy "basic" deleted 115 ``` 116 117 ## HTTPProxy API Specification 118 119 There are a number of [working examples][3] of HTTPProxy objects in the `examples/example-workload` directory. 120 121 We will use these examples as a mechanism to describe HTTPProxy API functionality. 122 123 ### Virtual Host Configuration 124 125 #### Fully Qualified Domain Name 126 127 Similar to Ingress, HTTPProxy support name-based virtual hosting. 128 Name-based virtual hosts use multiple host names with the same IP address. 129 130 ``` 131 foo.bar.com --| |-> foo.bar.com s1:80 132 | 178.91.123.132 | 133 bar.foo.com --| |-> bar.foo.com s2:80 134 ``` 135 136 Unlike Ingress, HTTPProxy only support a single root domain per HTTPProxy object. 137 As an example, this Ingress object: 138 139 ```yaml 140 # ingress-name.yaml 141 apiVersion: extensions/v1beta1 142 kind: Ingress 143 metadata: 144 name: name-example 145 spec: 146 rules: 147 - host: foo1.bar.com 148 http: 149 paths: 150 - backend: 151 serviceName: s1 152 servicePort: 80 153 - host: bar1.bar.com 154 http: 155 paths: 156 - backend: 157 serviceName: s2 158 servicePort: 80 159 ``` 160 161 must be represented by two different HTTPProxy objects: 162 163 ```yaml 164 # httpproxy-name.yaml 165 apiVersion: projectcontour.io/v1 166 kind: HTTPProxy 167 metadata: 168 name: name-example-foo 169 namespace: default 170 spec: 171 virtualhost: 172 fqdn: foo1.bar.com 173 routes: 174 - services: 175 - name: s1 176 port: 80 177 --- 178 apiVersion: projectcontour.io/v1 179 kind: HTTPProxy 180 metadata: 181 name: name-example-bar 182 namespace: default 183 spec: 184 virtualhost: 185 fqdn: bar1.bar.com 186 routes: 187 - services: 188 - name: s2 189 port: 80 190 ``` 191 192 #### TLS 193 194 HTTPProxy follows a similar pattern to Ingress for configuring TLS credentials. 195 196 You can secure a HTTPProxy by specifying a Secret that contains TLS private key and certificate information. 197 Contour (via Envoy) uses the SNI TLS extension to handle this behavior. 198 If multiple HTTPProxy's utilize the same Secret, the certificate must include the necessary Subject Authority Name (SAN) for each fqdn. 199 200 Contour also follows a "secure first" approach. 201 When TLS is enabled for a virtual host any request to the insecure port is redirected to the secure interface with a 301 redirect. 202 Specific routes can be configured to override this behavior and handle insecure requests by enabling the `spec.routes.permitInsecure` parameter on a Route. 203 204 The TLS secret must contain keys named tls.crt and tls.key that contain the certificate and private key to use for TLS, e.g.: 205 206 ```yaml 207 # ingress-tls.secret.yaml 208 apiVersion: v1 209 data: 210 tls.crt: base64 encoded cert 211 tls.key: base64 encoded key 212 kind: Secret 213 metadata: 214 name: testsecret 215 namespace: default 216 type: kubernetes.io/tls 217 ``` 218 219 The HTTPProxy can be configured to use this secret using `tls.secretName` property: 220 221 ```yaml 222 # httpproxy-tls.yaml 223 apiVersion: projectcontour.io/v1 224 kind: HTTPProxy 225 metadata: 226 name: tls-example 227 namespace: default 228 spec: 229 virtualhost: 230 fqdn: foo2.bar.com 231 tls: 232 secretName: testsecret 233 routes: 234 - services: 235 - name: s1 236 port: 80 237 ``` 238 239 If the `tls.secretName` property contains a slash, eg. `somenamespace/somesecret` then, subject to TLS Certificate Delegation, the TLS certificate will be read from `somesecret` in `somenamespace`. 240 See TLS Certificate Delegation below for more information. 241 242 The TLS **Minimum Protocol Version** a vhost should negotiate can be specified by setting the `spec.virtualhost.tls.minimumProtocolVersion`: 243 244 - 1.3 245 - 1.2 246 - 1.1 (Default) 247 248 #### Upstream TLS 249 250 A HTTPProxy can proxy to an upstream TLS connection by first annotating the upstream Kubernetes service with: `projectcontour.io/upstream-protocol.tls: "443,https"`. 251 This annotation tells Contour which port should be used for the TLS connection. 252 In this example, the upstream service is named `https` and uses port `443`. 253 Additionally, it is possible for Envoy to verify the backend service's certificate. 254 The service of an HTTPProxy can optionally specify a `validation` struct which has a mandatory `caSecret` key as well as an mandatory `subjectName`. 255 256 Note: If `spec.routes.services[].validation` is present, `spec.routes.services[].{name,port}` must point to a Service with a matching `projectcontour.io/upstream-protocol.tls` Service annotation. 257 258 ##### Sample YAML 259 260 ```yaml 261 apiVersion: projectcontour.io/v1 262 kind: HTTPProxy 263 metadata: 264 name: secure-backend 265 spec: 266 virtualhost: 267 fqdn: www.example.com 268 routes: 269 - services: 270 - name: service 271 port: 8443 272 validation: 273 caSecret: my-certificate-authority 274 subjectName: backend.example.com 275 ``` 276 277 ##### Error conditions 278 279 If the `validation` spec is defined on a service, but the secret which it references does not exist, Contour will reject the update and set the status of the HTTPProxy object accordingly. 280 This helps prevent the case of proxying to an upstream where validation is requested, but not yet available. 281 282 ```yaml 283 Status: 284 Current Status: invalid 285 Description: route "/": service "tls-nginx": upstreamValidation requested but secret not found or misconfigured 286 ``` 287 288 #### TLS Certificate Delegation 289 290 In order to support wildcard certificates, TLS certificates for a `*.somedomain.com`, which are stored in a namespace controlled by the cluster administrator, Contour supports a facility known as TLS Certificate Delegation. 291 This facility allows the owner of a TLS certificate to delegate, for the purposes of referencing the TLS certificate, permission to Contour to read the Secret object from another namespace. 292 293 The `TLSCertificateDelegation` resource defines a set of `delegations` in the `spec`. 294 Each delegation references a `secretName` from the namespace where the `TLSCertificateDelegation` is created as well as describing a set of `targetNamespaces` in which the certificate can be referenced. 295 If all namespaces should be able to reference the secret, then set `"*"` as the value of `targetNamespaces` (see example below). 296 297 ```yaml 298 apiVersion: projectcontour.io/v1 299 kind: TLSCertificateDelegation 300 metadata: 301 name: example-com-wildcard 302 namespace: www-admin 303 spec: 304 delegations: 305 - secretName: example-com-wildcard 306 targetNamespaces: 307 - example-com 308 - secretName: another-com-wildcard 309 targetNamespaces: 310 - "*" 311 --- 312 apiVersion: projectcontour.io/v1 313 kind: HTTPProxy 314 metadata: 315 name: www 316 namespace: example-com 317 spec: 318 virtualhost: 319 fqdn: foo2.bar.com 320 tls: 321 secretName: www-admin/example-com-wildcard 322 routes: 323 - services: 324 - name: s1 325 port: 80 326 ``` 327 328 In this example, the permission for Contour to reference the Secret `example-com-wildcard` in the `admin` namespace has been delegated to HTTPProxy objects in the `example-com` namespace. 329 Also, the permission for Contour to reference the Secret `another-com-wildcard` from all namespaces has been delegated to all HTTPProxy objects in the cluster. 330 331 ### Conditions 332 333 Each Route entry in a HTTPProxy **may** contain one or more conditions. 334 These conditions are combined with an AND operator on the route passed to Envoy. 335 336 Conditions can be either a `prefix` or a `header` condition. 337 338 #### Prefix conditions 339 340 For `prefix`, this adds a path prefix. 341 342 Up to one prefix condition may be present in any condition block. 343 344 Prefix conditions **must** start with a `/` if they are present. 345 346 #### Header conditions 347 348 For `header` conditions there is one required field, `name`, and five operator fields: `present`, `contains`, `notcontains`, `exact`, and `notexact`. 349 350 - `present` is a boolean and checks that the header is present. The value will not be checked. 351 352 - `contains` is a string, and checks that the header contains the string. `notcontains` similarly checks that the header does *not* contain the string. 353 354 - `exact` is a string, and checks that the header exactly matches the whole string. `notexact` checks that the header does *not* exactly match the whole string. 355 356 #### Multiple Routes 357 358 HTTPProxy must have at least one route or include defined. 359 Paths defined are matched using prefix conditions. 360 In this example, any requests to `multi-path.bar.com/blog` or `multi-path.bar.com/blog/*` will be routed to the Service `s2`. 361 All other requests to the host `multi-path.bar.com` will be routed to the Service `s1`. 362 363 ```yaml 364 # httpproxy-multiple-paths.yaml 365 apiVersion: projectcontour.io/v1 366 kind: HTTPProxy 367 metadata: 368 name: multiple-paths 369 namespace: default 370 spec: 371 virtualhost: 372 fqdn: multi-path.bar.com 373 routes: 374 - conditions: 375 - prefix: / # matches everything else 376 services: 377 - name: s1 378 port: 80 379 - conditions: 380 - prefix: /blog # matches `multi-path.bar.com/blog` or `multi-path.bar.com/blog/*` 381 services: 382 - name: s2 383 port: 80 384 ``` 385 386 In the following example, we match on headers and send to different services, with a default route if those do not match. 387 388 ```yaml 389 # httpproxy-multiple-headers.yaml 390 apiVersion: projectcontour.io/v1 391 kind: HTTPProxy 392 metadata: 393 name: multiple-paths 394 namespace: default 395 spec: 396 virtualhost: 397 fqdn: multi-path.bar.com 398 routes: 399 - conditions: 400 - header: 401 name: x-os 402 contains: ios 403 services: 404 - name: s1 405 port: 80 406 - conditions: 407 - header: 408 name: x-os 409 contains: android 410 services: 411 - name: s2 412 port: 80 413 - services: 414 - name: s3 415 port: 80 416 ``` 417 418 #### Multiple Upstreams 419 420 One of the key HTTPProxy features is the ability to support multiple services for a given path: 421 422 ```yaml 423 # httpproxy-multiple-upstreams.yaml 424 apiVersion: projectcontour.io/v1 425 kind: HTTPProxy 426 metadata: 427 name: multiple-upstreams 428 namespace: default 429 spec: 430 virtualhost: 431 fqdn: multi.bar.com 432 routes: 433 - services: 434 - name: s1 435 port: 80 436 - name: s2 437 port: 80 438 ``` 439 440 In this example, requests for `multi.bar.com/` will be load balanced across two Kubernetes Services, `s1`, and `s2`. 441 This is helpful when you need to split traffic for a given URL across two different versions of an application. 442 443 #### Upstream Weighting 444 445 Building on multiple upstreams is the ability to define relative weights for upstream Services. 446 This is commonly used for canary testing of new versions of an application when you want to send a small fraction of traffic to a specific Service. 447 448 ```yaml 449 # httpproxy-weight-shfiting.yaml 450 apiVersion: projectcontour.io/v1 451 kind: HTTPProxy 452 metadata: 453 name: weight-shifting 454 namespace: default 455 spec: 456 virtualhost: 457 fqdn: weights.bar.com 458 routes: 459 - services: 460 - name: s1 461 port: 80 462 weight: 10 463 - name: s2 464 port: 80 465 weight: 90 466 ``` 467 468 In this example, we are sending 10% of the traffic to Service `s1`, while Service `s2` receives the remaining 90% of traffic. 469 470 HTTPProxy weighting follows some specific rules: 471 472 - If no weights are specified for a given route, it's assumed even distribution across the Services. 473 - Weights are relative and do not need to add up to 100. If all weights for a route are specified, then the "total" weight is the sum of those specified. As an example, if weights are 20, 30, 20 for three upstreams, the total weight would be 70. In this example, a weight of 30 would receive approximately 42.9% of traffic (30/70 = .4285). 474 - If some weights are specified but others are not, then it's assumed that upstreams without weights have an implicit weight of zero, and thus will not receive traffic. 475 476 #### Traffic mirroring 477 478 Per route a service can be nominated as a mirror. 479 The mirror service will receive a copy of the read traffic sent to any non mirror service. 480 The mirror traffic is considered _read only_, any response by the mirror will be discarded. 481 482 This service can be useful for recording traffic for later replay or for smoke testing new deployments. 483 484 ```yaml 485 apiVersion: projectcontour.io/v1 486 kind: HTTPProxy 487 metadata: 488 name: traffic-mirror 489 namespace: default 490 spec: 491 virtualhost: 492 fqdn: www.example.com 493 routes: 494 - conditions: 495 - prefix: / 496 services: 497 - name: www 498 port: 80 499 - name: www-mirror 500 port: 80 501 mirror: true 502 ``` 503 504 #### Response Timeout 505 506 Each Route can be configured to have a timeout policy and a retry policy as shown: 507 508 ```yaml 509 # httpproxy-response-timeout.yaml 510 apiVersion: projectcontour.io/v1 511 kind: HTTPProxy 512 metadata: 513 name: response-timeout 514 namespace: default 515 spec: 516 virtualhost: 517 fqdn: timeout.bar.com 518 routes: 519 - timeoutPolicy: 520 response: 1s 521 idle: 10s 522 retryPolicy: 523 count: 3 524 perTryTimeout: 150ms 525 services: 526 - name: s1 527 port: 80 528 ``` 529 530 In this example, requests to `timeout.bar.com/` will have a response timeout policy of 1s. 531 This refers to the time that spans between the point at which complete client request has been processed by the proxy, and when the response from the server has been completely processed. 532 533 - `timeoutPolicy.response` This field can be any positive time period or "infinity". 534 The time period of **0s** will also be treated as infinity. 535 This timeout covers the time from the *end of the client request* to the *end of the upstream response*. 536 By default, Envoy has a 15 second value for this timeout. 537 More information can be found in [Envoy's documentation][4]. 538 - `timeoutPolicy.idle` This field can be any positive time period or "infinity". 539 The time period of **0s** will also be treated as infinity. 540 By default, there is no per-route idle timeout. 541 Note that the default connection manager idle timeout of 5 minutes will apply if this is not set. 542 543 TimeoutPolicy durations are expressed as per the format specified in the [ParseDuration documentation][5]. 544 Example input values: "300ms", "5s", "1m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". 545 The string 'infinity' is also a valid input and specifies no timeout. 546 547 More information can be found in [Envoy's documentation][6] 548 - `retryPolicy`: A retry will be attempted if the server returns an error code in the 5xx range, or if the server takes more than `retryPolicy.perTryTimeout` to process a request. 549 - `retryPolicy.count` specifies the maximum number of retries allowed. This parameter is optional and defaults to 1. 550 - `retryPolicy.perTryTimeout` specifies the timeout per retry. If this field is greater than the request timeout, it is ignored. This parameter is optional. 551 If left unspecified, `timeoutPolicy.request` will be used. 552 553 #### Load Balancing Strategy 554 555 Each route can have a load balancing strategy applied to determine which of its Endpoints is selected for the request. 556 The following list are the options available to choose from: 557 558 - `RoundRobin`: Each healthy upstream Endpoint is selected in round robin order (Default strategy if none selected). 559 - `WeightedLeastRequest`: The least request strategy uses an O(1) algorithm which selects two random healthy Endpoints and picks the Endpoint which has fewer active requests. Note: This algorithm is simple and sufficient for load testing. It should not be used where true weighted least request behavior is desired. 560 - `Random`: The random strategy selects a random healthy Endpoints. 561 562 More information on the load balancing strategy can be found in [Envoy's documentation][7]. 563 564 The following example defines the strategy for the route `/` as `WeightedLeastRequest`. 565 566 ```yaml 567 # httpproxy-lb-strategy.yaml 568 apiVersion: projectcontour.io/v1 569 kind: HTTPProxy 570 metadata: 571 name: lb-strategy 572 namespace: default 573 spec: 574 virtualhost: 575 fqdn: strategy.bar.com 576 routes: 577 - conditions: 578 - prefix: / 579 services: 580 - name: s1-strategy 581 port: 80 582 - name: s2-strategy 583 port: 80 584 loadBalancerPolicy: 585 strategy: WeightedLeastRequest 586 ``` 587 588 #### Session Affinity 589 590 Session affinity, also known as _sticky sessions_, is a load balancing strategy whereby a sequence of requests from a single client are consistently routed to the same application backend. 591 Contour supports session affinity on a per route basis with `loadBalancerPolocy` `strategy: Cookie`. 592 593 ```yaml 594 # httpproxy-sticky-sessions.yaml 595 apiVersion: projectcontour.io/v1 596 kind: HTTPProxy 597 metadata: 598 name: httpbin 599 namespace: default 600 spec: 601 virtualhost: 602 fqdn: httpbin.davecheney.com 603 routes: 604 - services: 605 - name: httpbin 606 port: 8080 607 loadBalancerPolicy: 608 strategy: Cookie 609 ``` 610 611 ##### Limitations 612 613 Session affinity is based on the premise that the backend servers are robust, do not change ordering, or grow and shrink according to load. 614 None of these properties are guaranteed by a Kubernetes cluster and will be visible to applications that rely heavily on session affinity. 615 616 Any perturbation in the set of pods backing a service risks redistributing backends around the hash ring. 617 618 #### Per route health checking 619 620 Active health checking can be configured on a per route basis. 621 Contour supports HTTP health checking and can be configured with various settings to tune the behavior. 622 623 During HTTP health checking Envoy will send an HTTP request to the upstream Endpoints. 624 It expects a 200 response if the host is healthy. 625 The upstream host can return 503 if it wants to immediately notify Envoy to no longer forward traffic to it. 626 It is important to note that these are health checks which Envoy implements and are separate from any other system such as those that exist in Kubernetes. 627 628 ```yaml 629 # httpproxy-health-checks.yaml 630 apiVersion: projectcontour.io/v1 631 kind: HTTPProxy 632 metadata: 633 name: health-check 634 namespace: default 635 spec: 636 virtualhost: 637 fqdn: health.bar.com 638 routes: 639 - conditions: 640 - prefix: / 641 healthCheckPolicy: 642 path: /healthy 643 intervalSeconds: 5 644 timeoutSeconds: 2 645 unhealthyThresholdCount: 3 646 healthyThresholdCount: 5 647 services: 648 - name: s1-health 649 port: 80 650 - name: s2-health 651 port: 80 652 ``` 653 654 Health check configuration parameters: 655 656 - `path`: HTTP endpoint used to perform health checks on upstream service (e.g. `/healthz`). It expects a 200 response if the host is healthy. The upstream host can return 503 if it wants to immediately notify downstream hosts to no longer forward traffic to it. 657 - `host`: The value of the host header in the HTTP health check request. If left empty (default value), the name "contour-envoy-healthcheck" will be used. 658 - `intervalSeconds`: The interval (seconds) between health checks. Defaults to 5 seconds if not set. 659 - `timeoutSeconds`: The time to wait (seconds) for a health check response. If the timeout is reached the health check attempt will be considered a failure. Defaults to 2 seconds if not set. 660 - `unhealthyThresholdCount`: The number of unhealthy health checks required before a host is marked unhealthy. Note that for http health checking if a host responds with 503 this threshold is ignored and the host is considered unhealthy immediately. Defaults to 3 if not defined. 661 - `healthyThresholdCount`: The number of healthy health checks required before a host is marked healthy. Note that during startup, only a single successful health check is required to mark a host healthy. 662 663 #### WebSocket Support 664 665 WebSocket support can be enabled on specific routes using the `enableWebsockets` field: 666 667 ```yaml 668 # httpproxy-websockets.yaml 669 apiVersion: projectcontour.io/v1 670 kind: HTTPProxy 671 metadata: 672 name: chat 673 namespace: default 674 spec: 675 virtualhost: 676 fqdn: chat.example.com 677 routes: 678 - services: 679 - name: chat-app 680 port: 80 681 - conditions: 682 - prefix: /websocket 683 enableWebsockets: true # Setting this to true enables websocket for all paths that match /websocket 684 services: 685 - name: chat-app 686 port: 80 687 ``` 688 689 #### Permit Insecure 690 691 A HTTPProxy can be configured to permit insecure requests to specific Routes. 692 In this example, any request to `foo2.bar.com/blog` will not receive a 301 redirect to HTTPS, but the `/` route will: 693 694 ```yaml 695 apiVersion: projectcontour.io/v1 696 kind: HTTPProxy 697 metadata: 698 name: tls-example-insecure 699 namespace: default 700 spec: 701 virtualhost: 702 fqdn: foo2.bar.com 703 tls: 704 secretName: testsecret 705 routes: 706 - services: 707 - name: s1 708 port: 80 709 - conditions: 710 - prefix: /blog 711 permitInsecure: true 712 services: 713 - name: s2 714 port: 80 715 ``` 716 717 #### ExternalName 718 719 HTTPProxy supports routing traffic to service types `ExternalName`. 720 Contour looks at the `spec.externalName` field of the service and configures the route to use that DNS name instead of utilizing EDS. 721 722 There's nothing specific in the HTTPProxy object that needs to be configured other than referencing a service of type `ExternalName`. 723 724 NOTE: The ports are required to be specified. 725 726 ```yaml 727 # httpproxy-externalname.yaml 728 apiVersion: v1 729 kind: Service 730 metadata: 731 labels: 732 run: externaldns 733 name: externaldns 734 namespace: default 735 spec: 736 externalName: foo-basic.bar.com 737 ports: 738 - name: http 739 port: 80 740 protocol: TCP 741 targetPort: 80 742 type: ExternalName 743 ``` 744 745 ## HTTPProxy inclusion 746 747 HTTPProxy permits the splitting of a system's configuration into separate HTTPProxy instances using **inclusion**. 748 749 Inclusion, as the name implies, allows for one HTTPProxy object to be included in another, optionally with some conditions inherited from the parent. 750 Contour reads the inclusion tree and merges the included routes into one big object internally before rendering Envoy config. 751 Importantly, the included HTTPProxy objects do not have to be in the same namespace, so this is functionally the same as the delegation feature of the now-deprecated IngressRoute. 752 753 Each tree of HTTPProxy starts with a root, the top level object of the configuration for a particular virtual host. 754 Each root HTTPProxy defines a `virtualhost` key, which describes properties such as the fully qualified name of the virtual host, TLS configuration, etc. 755 756 HTTPProxies included from the root must not contain a virtualhost key. 757 Root objects cannot include other roots either transitively or directly. 758 This permits the owner of an HTTPProxy root to allow the inclusion of a portion of the route space inside a virtual host, and to allow that route space to be further subdivided with inclusions. 759 Because the path is not necessarily used as the only key, the route space can be multi-dimensional. 760 761 ### Conditions and Inclusion 762 763 Like Routes, Inclusion may specify a set of [conditions][8]. 764 These conditions are added to any conditions on the routes included. 765 This process is recursive. 766 767 Conditions are sets of individual condition statements, for example `prefix: /blog` is the condition that the matching request's path must start with `/blog`. 768 When conditions are combined through inclusion Contour merges the conditions inherited via inclusion with any conditions specified on the route. 769 This may result in duplicates, for example two `prefix:` conditions, or two header match conditions with the same name and value. 770 To resolve this Contour applies the following logic. 771 772 - `prefix:` conditions are concatenated together in the order they were applied from the root object. For example the conditions, `prefix: /api`, `prefix: /v1` becomes a single `prefix: /api/v1` conditions. Note: Multiple prefixes cannot be supplied on a single set of Route conditions. 773 - Proxies with repeated identical `header:` conditions of type "exact match" (the same header keys exactly) are marked as "Invalid" since they create an un-routable configuration. 774 775 ### Configuring inclusion 776 777 Inclusion is a top-level part of the HTTPProxy `spec` element. 778 It requires one field, `name`, and has two optional fields: 779 780 - `namespace`. This will assume the included HTTPProxy is in the same namespace if it's not specified. 781 - a `conditions` block. 782 783 #### Within the same namespace 784 785 HTTPProxies can include other HTTPProxy objects in the namespace by specifying the name of the object and its namespace in the top-level `includes` block. 786 Note that `includes` is a list, and so it must use the YAML list construct. 787 788 In this example, the HTTPProxy `include-root` has included the configuration for paths matching `/service2` from the HTTPPRoxy named `service2` in the same namespace as `include-root` (the `default` namespace). 789 It's important to note that `service2` HTTPProxy has not defined a `virtualhost` property as it is NOT a root HTTPProxy. 790 791 ```yaml 792 # httpproxy-inclusion-samenamespace.yaml 793 apiVersion: projectcontour.io/v1 794 kind: HTTPProxy 795 metadata: 796 name: include-root 797 namespace: default 798 spec: 799 virtualhost: 800 fqdn: root.bar.com 801 includes: 802 # Includes the /service2 path from service2 in the same namespace 803 - name: www 804 namespace: default 805 conditions: 806 - prefix: /service2 807 routes: 808 - conditions: 809 - prefix: / 810 services: 811 - name: s1 812 port: 80 813 --- 814 apiVersion: projectcontour.io/v1 815 kind: HTTPProxy 816 metadata: 817 name: service2 818 namespace: default 819 spec: 820 routes: 821 - conditions: 822 - prefix: / # matches /service2 823 services: 824 - name: s2 825 port: 80 826 - conditions: 827 - prefix: /blog # matches /service2/blog 828 services: 829 - name: blog 830 port: 80 831 ``` 832 833 #### Virtualhost aliases 834 835 To present the same set of routes under multiple dns entries, for example www.example.com and example.com, including a service with a `prefix` condition of `/` can be used. 836 837 ```yaml 838 # httpproxy-inclusion-multipleroots.yaml 839 --- 840 apiVersion: projectcontour.io/v1 841 kind: HTTPProxy 842 metadata: 843 name: multiple-root 844 namespace: default 845 spec: 846 virtualhost: 847 fqdn: bar.com 848 includes: 849 - name: main 850 namespace: default 851 --- 852 apiVersion: projectcontour.io/v1 853 kind: HTTPProxy 854 metadata: 855 name: multiple-root-www 856 namespace: default 857 spec: 858 virtualhost: 859 fqdn: www.bar.com 860 includes: 861 - name: main 862 namespace: default 863 --- 864 apiVersion: projectcontour.io/v1 865 kind: HTTPProxy 866 metadata: 867 name: main 868 namespace: default 869 spec: 870 routes: 871 - services: 872 - name: s2 873 port: 80 874 ``` 875 876 #### Across namespaces 877 878 Inclusion can also happen across Namespaces by specifying a `namespace` in the `inclusion`. 879 This is a particularly powerful paradigm for enabling multi-team Ingress management. 880 881 In this example, the root HTTPProxy has included configuration for paths matching `/blog` to the `blog` HTTPProxy object in the `marketing` namespace. 882 883 ```yaml 884 # httpproxy-inclusion-across-namespaces.yaml 885 --- 886 apiVersion: projectcontour.io/v1 887 kind: HTTPProxy 888 metadata: 889 name: namespace-include-root 890 namespace: default 891 spec: 892 virtualhost: 893 fqdn: ns-root.bar.com 894 includes: 895 # delegate the subpath, `/blog` to the IngressRoute object in the marketing namespace with the name `blog` 896 - name: blog 897 namespace: marketing 898 conditions: 899 - prefix: /blog 900 routes: 901 - services: 902 - name: s1 903 port: 80 904 905 --- 906 apiVersion: projectcontour.io/v1 907 kind: HTTPProxy 908 metadata: 909 name: blog 910 namespace: marketing 911 spec: 912 routes: 913 - services: 914 - name: s2 915 port: 80 916 ``` 917 918 ### Orphaned HTTPProxy children 919 920 It is possible for HTTPProxy objects to exist that have not been delegated to by another HTTPProxy. 921 These objects are considered "orphaned" and will be ignored by Contour in determining ingress configuration. 922 923 ### Restricted root namespaces 924 925 HTTPProxy inclusion allows for Administrators to limit which users/namespaces may configure routes for a given domain, but it does not restrict where root HTTPProxy may be created. 926 Contour has an enforcing mode which accepts a list of namespaces where root HTTPProxy are valid. 927 Only users permitted to operate in those namespaces can therefore create HTTPProxy with the `virtualhost` field. 928 929 This restricted mode is enabled in Contour by specifying a command line flag, `--root-namespaces`, which will restrict Contour to only searching the defined namespaces for root HTTPProxy. This CLI flag accepts a comma separated list of namespaces where HTTPProxy are valid (e.g. `--root-namespaces=default,kube-system,my-admin-namespace`). 930 931 HTTPProxy with a defined `virtualhost` field that are not in one of the allowed root namespaces will be flagged as `invalid` and will be ignored by Contour. 932 933 Additionally, when defined, Contour will only watch for Kubernetes secrets in these namespaces ignoring changes in all other namespaces. 934 Proper RBAC rules should also be created to restrict what namespaces Contour has access matching the namespaces passed to the command line flag. 935 An example of this is included in the [examples directory][1] and shows how you might create a namespace called `root-httproxies`. 936 937 > **NOTE: The restricted root namespace feature is only supported for HTTPProxy CRDs. 938 > `--root-namespaces` does not affect the operation of `v1beta1.Ingress` objects** 939 940 ## TCP Proxying 941 942 HTTPProxy supports proxying of TLS encapsulated TCP sessions. 943 944 _Note_: The TCP session must be encrypted with TLS. 945 This is necessary so that Envoy can use SNI to route the incoming request to the correct service. 946 947 ### TLS Termination at the edge 948 949 If `spec.virtualhost.tls.secretName` is present then that secret will be used to decrypt the TCP traffic at the edge. 950 951 ```yaml 952 # httpproxy-tls-termination.yaml 953 apiVersion: projectcontour.io/v1 954 kind: HTTPProxy 955 metadata: 956 name: example 957 namespace: default 958 spec: 959 virtualhost: 960 fqdn: tcp.example.com 961 tls: 962 secretName: secret 963 tcpproxy: 964 services: 965 - name: tcpservice 966 port: 8080 967 - name: otherservice 968 port: 9999 969 weight: 20 970 ``` 971 972 The `spec.tcpproxy` key indicates that this _root_ HTTPProxy will forward the de-encrypted TCP traffic to the backend service. 973 974 ### TLS passthrough to the backend service 975 976 If you wish to handle the TLS handshake at the backend service set `spec.virtualhost.tls.passthrough: true` indicates that once SNI demuxing is performed, the encrypted connection will be forwarded to the backend service. 977 The backend service is expected to have a key which matches the SNI header received at the edge, and be capable of completing the TLS handshake. This is called SSL/TLS Passthrough. 978 979 ```yaml 980 # httpproxy-tls-passthrough.yaml 981 apiVersion: projectcontour.io/v1 982 kind: HTTPProxy 983 metadata: 984 name: example 985 namespace: default 986 spec: 987 virtualhost: 988 fqdn: tcp.example.com 989 tls: 990 passthrough: true 991 tcpproxy: 992 services: 993 - name: tcpservice 994 port: 8080 995 - name: otherservice 996 port: 9999 997 weight: 20 998 ``` 999 1000 ### TCPProxy delegation 1001 1002 There can be at most one TCPProxy stanza per root HTTPProxy, however that TCPProxy does not need to be defined in the root HTTPProxy object. 1003 HTTPProxy authors can delegate the configuration of a TCPProxy to the TCPProxy configuration defined in a HTTPProxy child object. 1004 1005 ```yaml 1006 # httpproxy-parent-termination.yaml 1007 apiVersion: projectcontour.io/v1 1008 kind: HTTPProxy 1009 metadata: 1010 name: parent 1011 namespace: default 1012 spec: 1013 virtualhost: 1014 fqdn: tcp.example.com 1015 tls: 1016 secretName: secret 1017 tcpproxy: 1018 include: 1019 name: child 1020 namespace: app 1021 --- 1022 # httpproxy-child-termination.yaml 1023 apiVersion: projectcontour.io/v1 1024 kind: HTTPProxy 1025 metadata: 1026 name: child 1027 namespace: app 1028 spec: 1029 tcpproxy: 1030 services: 1031 - name: tcpservice 1032 port: 8080 1033 - name: otherservice 1034 port: 9999 1035 weight: 20 1036 ``` 1037 In this example `default/parent` delegates the configuration of the TCPProxy services to `app/child`. 1038 1039 ## Upstream Validation 1040 1041 When defining upstream services on a route, it's possible to configure the connection from Envoy to the backend endpoint to communicate over TLS. 1042 Two configuration items are required, a CA certificate and a `SubjectName` which are both used to verify the backend endpoint's identity. 1043 1044 The CA certificate bundle for the backend service should be supplied in a Kubernetes Secret. 1045 The referenced Secret must be of type "Opaque" and have a data key named `ca.crt`. 1046 This data value must be a PEM-encoded certificate bundle. 1047 1048 In addition to the CA certificate and the subject name, the Kubernetes service must also be annotated with a Contour specific annotation: `projectcontour.io/upstream-protocol.tls: <port>` ([see annotations section][9]) 1049 1050 _Note: This annotation is applied to the Service not the Ingress or HTTPProxy object._ 1051 1052 ```yaml 1053 apiVersion: projectcontour.io/v1 1054 kind: HTTPProxy 1055 metadata: 1056 name: blog 1057 namespace: marketing 1058 spec: 1059 routes: 1060 - services: 1061 - name: s2 1062 port: 80 1063 validation: 1064 caSecret: foo-ca-cert 1065 subjectName: foo.marketing 1066 ``` 1067 1068 ## Status Reporting 1069 1070 There are many misconfigurations that could cause an HTTPProxy or delegation to be invalid. 1071 To aid users in resolving these issues, Contour updates a `status` field in all HTTPProxy objects. 1072 In the current specification, invalid HTTPProxy are ignored by Contour and will not be used in the ingress routing configuration. 1073 1074 If an HTTPProxy object is valid, it will have a status property that looks like this: 1075 1076 ```yaml 1077 status: 1078 currentStatus: valid 1079 description: valid HTTPProxy 1080 ``` 1081 1082 If the HTTPProxy is invalid, the `currentStatus` field will be `invalid` and the `description` field will provide a description of the issue. 1083 1084 As an example, if an HTTPProxy object has specified a negative value for weighting, the HTTPProxy status will be: 1085 1086 ```yaml 1087 status: 1088 currentStatus: invalid 1089 description: "route '/foo': service 'home': weight must be greater than or equal to zero" 1090 ``` 1091 1092 Some examples of invalid configurations that Contour provides statuses for: 1093 1094 - Negative weight provided in the route definition. 1095 - Invalid port number provided for service. 1096 - Prefix in parent does not match route in delegated route. 1097 - Root HTTPProxy created in a namespace other than the allowed root namespaces. 1098 - A given Route of an HTTPProxy both delegates to another HTTPProxy and has a list of services. 1099 - Orphaned route. 1100 - Delegation chain produces a cycle. 1101 - Root HTTPProxy does not specify fqdn. 1102 - Multiple prefixes cannot be specified on the same set of route conditions. 1103 - Multiple header conditions of type "exact match" with the same header key. 1104 1105 [1]: https://kubernetes.io/docs/concepts/services-networking/ingress/ 1106 [2]: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md 1107 [3]: {{< param github_url >}}/tree/{{page.version}}/examples/example-workload/httpproxy 1108 [4]: https://www.envoyproxy.io/docs/envoy/v1.11.2/api-v2/api/v2/route/route.proto.html#envoy-api-field-route-routeaction-timeout 1109 [5]: https://godoc.org/time#ParseDuration 1110 [6]: https://www.envoyproxy.io/docs/envoy/v1.11.2/api-v2/api/v2/route/route.proto.html#envoy-api-field-route-routeaction-idle-timeout 1111 [7]: https://www.envoyproxy.io/docs/envoy/v1.11.2/intro/arch_overview/upstream/load_balancing/overview 1112 [8]: #conditions 1113 [9]: {% link docs/{{page.version}}/annotations.md %}