github.com/openshift/installer@v1.4.17/docs/user/agent/add-node/add-nodes.md (about) 1 # Adding a node via the node-joiner tool 2 3 ## Pre-requisites 4 1. The `oc` tool must be available in the execution environment (the "user host"). 5 2. Ensure `oc` is properly configured to manage the cluster to be expanded, i.e. by either: 6 - running `oc login` command with the required cluster credentials, or 7 - running `export KUBECONFIG=<cluster `kubeconfig` file>` 8 9 ## Setup 10 1. Download the [node-joiner.sh](./node-joiner.sh) script in a working directory in 11 the user host (the "assets folder"). 12 2. Create a `nodes-config.yaml` in the assets folder. This configuration file must contain the 13 list of all the nodes that the user wants to add to the target cluster. At minimum, the name and primary interface MAC address must be specified. For example: 14 ``` 15 hosts: 16 - hostname: extra-worker-0 17 interfaces: 18 - name: eth0 19 macAddress: 00:02:46:e3:9e:7c 20 - hostname: extra-worker-1 21 interfaces: 22 - name: eth0 23 macAddress: 00:02:46:e3:9e:8c 24 - hostname: extra-worker-2 25 interfaces: 26 - name: eth0 27 macAddress: 00:02:46:e3:9e:9c 28 ``` 29 3. Optionally, it's possible to specify - for each node - an `NMState` configuration block denoted below as `networkConfig` 30 (it will be applied during the first boot), for example: 31 ``` 32 hosts: 33 - hostname: extra-worker-0 34 interfaces: 35 - name: eth0 36 macAddress: 00:02:46:e3:9e:7c 37 networkConfig: 38 interfaces: 39 - name: eth0 40 type: ethernet 41 state: up 42 mac-address: 00:02:46:e3:9e:7c 43 ipv4: 44 enabled: true 45 address: 46 - ip: 192.168.111.90 47 prefix-length: 24 48 dhcp: false 49 dns-resolver: 50 config: 51 server: 52 - 192.168.111.1 53 routes: 54 config: 55 - destination: 0.0.0.0/0 56 next-hop-address: 192.168.111.1 57 next-hop-interface: eth0 58 table-id: 254 59 - hostname: extra-worker-1 60 interfaces: 61 - name: eth0 62 macAddress: 00:02:46:e3:9e:8c 63 - hostname: extra-worker-2 64 interfaces: 65 - name: eth0 66 macAddress: 00:02:46:e3:9e:9c 67 ``` 68 69 ## ISO generation 70 Run [node-joiner.sh](./node-joiner.sh): 71 ```bash 72 $ ./node-joiner.sh 73 ``` 74 The script will generate a temporary namespace prefixed with `openshift-node-joiner` in the target cluster, 75 where a pod will be launched to execute the effective node-joiner workload. 76 In case of success, the `node.x86_64.iso` ISO image will be downloaded in the assets folder. 77 78 ### Configuration file name 79 By default the script looks for a configuration file named `nodes-config.yaml`. It's possible to specify a 80 different config file name, as the first parameter of the script: 81 82 ```bash 83 $ ./node-joiner.sh config.yaml 84 ``` 85 86 ## Nodes joining 87 Use the iso image to boot all the nodes listed in the configuration file, and wait for the related 88 certificate signing requests (CSRs) to appear. When adding a new node to the cluster, two pending CSRs will 89 be generated, and they must be manually approved by the user. 90 91 Use the following command or [node-joiner-monitor.sh](./node-joiner-monitor.sh) described below to monitor the pending certificates: 92 ``` 93 $ oc get csr 94 ``` 95 Use the `oc` `approve` command to approve them: 96 ``` 97 $ oc adm certificate approve <csr_name> 98 ``` 99 Once all the pendings certificates will be approved, then the new node will become available: 100 ``` 101 $ oc get nodes 102 NAME STATUS ROLES AGE VERSION 103 extra-worker-0 Ready worker 1h v1.29.3+8628c3c 104 master-0 Ready control-plane,master 31h v1.29.3+8628c3c 105 master-1 Ready control-plane,master 32h v1.29.3+8628c3c 106 master-2 Ready control-plane,master 32h v1.29.3+8628c3c 107 ``` 108 109 # Monitoring 110 After a node is booted using the ISO image, progress can be monitored using the node-joiner-monitor.sh script. 111 112 Download the [node-joiner-monitor.sh](./node-joiner-monitor.sh) script to a local directory. 113 114 The script requires the IP address of the node to monitor. 115 116 Run [node-joiner-monitor.sh](./node-joiner-monitor.sh): 117 ```bash 118 $ ./node-joiner-monitor.sh 192.168.111.90 119 ``` 120 121 The script will execute a command to monitor the node using a temporary namespace with 122 prefix `openshift-node-joiner-monitor` in the target cluster. The output of this command 123 is printed out to stdout. 124 125 The script shows useful information about the node as it joins the cluster. 126 * Pre-flight validations. In case the node does not pass one or more validations, the installation will not start. The output of the failed validations are reported to allow users to fix the problem(s) when required. 127 * Installation progress indicating the current stage is shown. For example, writing of the image to disk, and initial reboot are reported. 128 * CSRs requiring the user's approval are shown. 129 130 The script exits either after the node has joined the cluster and is in ready state or after 90 minutes have elapsed. 131 132 Sample monitoring output: 133 ``` 134 INFO[2024-04-29T22:45:39-04:00] Monitoring IPs: [192.168.111.90] 135 INFO[2024-04-29T22:48:17-04:00] Node 192.168.111.90: Assisted Service API is available 136 INFO[2024-04-29T22:48:17-04:00] Node 192.168.111.90: Cluster is adding hosts 137 INFO[2024-04-29T22:48:17-04:00] Node 192.168.111.90: Updated image information (Image type is "full-iso", SSH public key is set) 138 INFO[2024-04-29T22:48:22-04:00] Node 192.168.111.90: Host ca241aa5-4f86-42bf-95a3-6b7ab7d4d66a: Successfully registered 139 WARNING[2024-04-29T22:48:32-04:00] Node 192.168.111.90: Host couldn't synchronize with any NTP server 140 WARNING[2024-04-29T22:48:32-04:00] Node 192.168.111.90: Host extraworker-0: updated status from discovering to insufficient (Host does not meet the minimum hardware requirements: Host couldn't synchronize with any NTP server) 141 INFO[2024-04-29T22:49:28-04:00] Node 192.168.111.90: Host extraworker-0: updated status from known to installing (Installation is in progress) 142 INFO[2024-04-29T22:50:28-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 5% 143 INFO[2024-04-29T22:50:33-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 16% 144 INFO[2024-04-29T22:50:38-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 28% 145 INFO[2024-04-29T22:50:43-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 40% 146 INFO[2024-04-29T22:50:48-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 51% 147 INFO[2024-04-29T22:50:53-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 67% 148 INFO[2024-04-29T22:50:58-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 77% 149 INFO[2024-04-29T22:51:03-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 88% 150 INFO[2024-04-29T22:51:08-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Writing image to disk: 93% 151 INFO[2024-04-29T22:51:13-04:00] Node 192.168.111.90: Host: extraworker-0, reached installation stage Rebooting 152 INFO[2024-04-29T22:56:35-04:00] Node 192.168.111.90: Kubelet is running 153 INFO[2024-04-29T22:56:45-04:00] Node 192.168.111.90: First CSR Pending approval 154 INFO[2024-04-29T22:56:45-04:00] CSR csr-257ms with signerName kubernetes.io/kube-apiserver-client-kubelet and username system:serviceaccount:openshift-machine-config-operator:node-bootstrapper is Pending and awaiting approval 155 INFO[2024-04-29T22:58:50-04:00] Node 192.168.111.90: Second CSR Pending approval 156 INFO[2024-04-29T22:58:50-04:00] CSR csr-tc8xt with signerName kubernetes.io/kubelet-serving and username system:node:extraworker-0 is Pending and awaiting approval 157 INFO[2024-04-29T22:58:50-04:00] Node 192.168.111.90: Node joined cluster 158 INFO[2024-04-29T23:00:00-04:00] Node 192.168.111.90: Node is Ready 159 ``` 160