github.com/interconnectedcloud/qdr-operator@v0.0.0-20210826174505-576d2b33dac7/README.md (about)

     1  # QDR Operator
     2  
     3  A Kubernetes operator for managing Layer 7 (e.g. Application Layer) addressing and routing within and across clusters. The operator manages *interior* and *edge* QDR deployments automating resource creation and administration.
     4  
     5  ## Introduction
     6  
     7  This operator provides an `Interconnect` [Custom Resource Definition](https://kubernetes.io/docs/tasks/access-kubernetes-api/extend-api-custom-resource-definitions/)
     8  (CRD) that describes a deployment of [Apache Qpid Dispatch Router](https://qpid.apache.org/components/dispatch-router/index.html) allowing developers to expertly
     9  deploy routers for their infrastructure and middle-ware oriented  messaging requirements. The number of messaging routers, the deployment topology, address
    10  semantics and other desired options can be specified through the CRD.
    11  
    12  ## Usage
    13  
    14  Deploy the QDR Operator into the Kubernetes cluster where it will manage requests for the `Interconnect` resource. The QDR Operator will watch for create, update and delete resource requests and perform the necessary steps to ensure the present cluster state matches the desired state.
    15  
    16  ### Deploy QDR Operator
    17  
    18  The `deploy` directory contains the manifests needed to properly install the
    19  Operator.
    20  
    21  Create the service account for the operator.
    22  
    23  ```
    24  $ kubectl create -f deploy/service_account.yaml
    25  ```
    26  
    27  Create the RBAC role and role-binding that grants the permissions
    28  necessary for the operator to function.
    29  
    30  ```
    31  $ kubectl create -f deploy/role.yaml
    32  $ kubectl create -f deploy/role_binding.yaml
    33  ```
    34  
    35  Deploy the CRD to the cluster that defines the Interconnect resource.
    36  
    37  ```
    38  $ kubectl create -f deploy/crds/interconnectedcloud_v1alpha1_interconnect_crd.yaml
    39  ```
    40  
    41  Next, deploy the operator into the cluster.
    42  
    43  ```
    44  $ kubectl create -f deploy/operator.yaml
    45  ```
    46  
    47  This step will create a pod on the Kubernetes cluster for the QDR Operator.
    48  Observe the `qdr-operator` pod and verify it is in the running state.
    49  
    50  ```
    51  $ kubectl get pods -l name=qdr-operator
    52  ```
    53  
    54  If for some reason, the pod does not get to the running state, look at the
    55  pod details to review any event that prohibited the pod from starting.
    56  
    57  ```
    58  $ kubectl describe pod -l name=qdr-operator
    59  ```
    60  
    61  You will be able to confirm that the new CRD has been registered in the cluster and you can review its details.
    62  
    63  ```
    64  $ kubectl get crd
    65  $ kubectl describe crd interconnects.interconnectedcloud.github.io
    66  ```
    67  
    68  To create a router deployment, you must create a `Interconnect` resource representing the desired specification of the deployment. For example, to create a 3-node router mesh deployment you may run:
    69  
    70  ```console
    71  $ cat <<EOF | kubectl create -f -
    72  apiVersion: interconnectedcloud.github.io/v1alpha1
    73  kind: Interconnect
    74  metadata:
    75    name: example-interconnect
    76  spec:
    77    # Add fields here
    78    deploymentPlan:
    79      image: quay.io/interconnectedcloud/qdrouterd:1.9.0
    80      role: interior
    81      size: 3
    82      placement: Any
    83  EOF
    84  ```
    85  
    86  The operator will create a deployment of three router instances, all connected together with default address semantics. It will also create a service through which the *interior* router mesh can be accessed. It will configure a default set of *listeners* and *connectors* as described below. You will be able to confirm that the instance has been created in the cluster and you can review its details. To view the Interconnect instance, the deployment it manages and the associated pods that are deployed:
    87  
    88  ```
    89  $ kubectl describe interconnect example-interconnect
    90  $ kubectl describe deploy example-interconnect
    91  $ kubectl describe svc example-interconnect
    92  $ kubectl get pods -o yaml
    93  ```
    94  
    95  ### Deployment Plan
    96  
    97  The CRD *Deployment Plan* defines the attributes for an Interconnect instance.
    98  
    99  #### Role and Placement
   100  
   101  The *Deployment Plan* **Role** defines the mode of operation for the routers in a topology.
   102  
   103    * **interior** - This role creates an interconnect of auto-meshed routers for concurrent connection capacity and resiliency.
   104      Connectivity between the routers will be defined by *InterRouterListeners* and *InterRouterConnectors*. Downlink
   105      connectivity with *edge* routers will be via *EdgeListeners*.
   106  
   107    * **edge** -  This role creates a set of stand-alone routers. The connectivity from the *edge* to *interior* routers
   108      will be via *EdgeConnectors*.
   109  
   110  The *Deployment Plan* **Placement** defines the deployment resource and the associated scheduling of the pods in the cluster.
   111  
   112    * **Any** - There is no constraint on pod placement. The operator will manage a *Deployment* resource where the number of
   113      pods scheduled will be up to *Deployment Plan Size*.
   114  
   115    * **Every** - Router pods will be placed on each node in the cluster. The operator will manage a *DaemonSet* resource where the
   116      number of pods scheduled will correspond to the number of nodes in the cluster. Note the *Deployment Plan Size* is
   117      not used.
   118  
   119    * **Anti-Affinity** - This constrains scheduling and prevents multiple router pods from running on the same node in the
   120      cluster. The operator will manage a *Deployment* resource with number of pods up to *Deployment Plan Size*.
   121      Note if *Deployment Plan Size* is greater than the number of nodes in the cluster, the excess pods that cannot be
   122      scheduled will remain in the *pending* state.
   123  
   124  ### Connectivity
   125  
   126  The connectivity between routers in a deployment is via the declared *listeners* and *connectors*. There are three types of *listeners* supported by the operator.
   127  
   128    * **Listeners** - A listener for accepting normal messaging client connections. The operator supports this listener for
   129      both *interior* and *edge* routers.
   130  
   131    * **InterRouterListeners** - A listener for accepting connections from peer *interior* routers. The operator
   132      support this listener for *interior* routers **only**.
   133  
   134    * **EdgeListeners** - A listener for accepting connections from downlink *edge* routers. The operator supports this
   135      listener for *interior* routers **only**.
   136  
   137  There are three types of *connectors* supported by the operator.
   138  
   139    * **Connectors** - A connector for connecting to an external messaging intermediary. The operator supports this connector
   140      for both *interior* and *edge* routers.
   141  
   142    * **InterRouterConnectors** - A connector for establishing connectivity to peer *interior* routers. The operator
   143      supports this connector for *interior* routers **only**.
   144  
   145    * **EdgeConnector** - A connector for establishing up-link connectivity from *edge* to *interior* routers. The operator
   146      supports this connector for *edge* routers **only**.
   147  
   148  ## Development
   149  
   150  This Operator is built using the [Operator SDK](https://github.com/operator-framework/operator-sdk). Follow the [Quick Start](https://github.com/operator-framework/operator-sdk) instructions to checkout and install the operator-sdk CLI.
   151  
   152  Local development may be done with [minikube](https://github.com/kubernetes/minikube) or [minishift](https://www.okd.io/minishift/).
   153  
   154  #### Source Code
   155  
   156  Clone this repository to a location on your workstation such as `$GOPATH/src/github.com/ORG/REPO`. Navigate to the repository and install the dependencies.
   157  
   158  ```
   159  $ cd $GOPATH/src/github.com/ORG/REPO/qdr-operator
   160  $ dep ensure && dep status
   161  ```
   162  
   163  #### Run Operator Locally
   164  
   165  Ensure the service account, role, role bindings and CRD are added to  the local cluster.
   166  
   167  ```
   168  $ kubectl create -f deploy/service_account.yaml
   169  $ kubectl create -f deploy/role.yaml
   170  $ kubectl create -f deploy/role_binding.yaml
   171  $ kubectl create -f deploy/crds/interconnectedcloud_v1alpha1_interconnect_crd.yaml
   172  ```
   173  
   174  Start the operator locally for development.
   175  
   176  ```
   177  $ operator-sdk up local
   178  ```
   179  
   180  Create a minimal Interconnect resource to observe and test your changes.
   181  
   182  ```console
   183  $ cat <<EOF | kubectl create -f -
   184  apiVersion: interconnectedcloud.github.io/v1alpha1
   185  kind: Interconnect
   186  metadata:
   187    name: example-interconnect
   188  spec:
   189    deploymentPlan:
   190      image: quay.io/interconnectedcloud/qdrouterd:1.9.0
   191      role: interior
   192      size: 3
   193      placement: Any
   194  EOF
   195  ```
   196  
   197  As you make local changes to the code, restart the operator to enact the changes.
   198  
   199  #### Build
   200  
   201  The Makefile will do the dependency check, operator-sdk generate k8s, run local test, and finally the operator-sdk build. Please ensure any local docker server is running.
   202  
   203  ```
   204  make
   205  ```
   206  
   207  #### Test
   208  
   209  Before submitting PR, please test your code. 
   210  
   211  File or local validation.
   212  ```
   213  $ make test
   214  ```
   215  
   216  Cluster-based test. 
   217  Ensure there is a cluster running before running the test.
   218  
   219  ```
   220  $ make cluster-test
   221  ```
   222  
   223  ## Manage the operator using the Operator Lifecycle Manager
   224  
   225  Ensure the Operator Lifecycle Manager is installed in the local cluster.  By default, the `catalog-source.sh` will install the operator catalog resources in `operator-lifecycle-manager` namespace.  You may also specify different namespace where you have the Operator Lifecycle Manager installed.
   226  
   227  ```
   228  $ ./hack/catalog-source.sh [namespace]
   229  $ oc apply -f deploy/olm-catalog/qdr-operator/0.1.0/catalog-source.yaml
   230  ```