github.com/openshift/installer@v1.4.17/upi/vsphere/README.md (about) 1 This project shows two ways to install an UPI cluster. We will discuss how to install using one of these two techniques: 2 - Terraform 3 - PowerShell 4 5 # Table of Contents 6 - [PowerShell](#PowerShell) 7 - [Pre-Requisites](#pre-requisites) 8 - [PowerShell Setup](#powershell-setup) 9 - [VMware.PowerCLI](#vmwarepowercli) 10 - [EPS](#eps) 11 - [Script Configuration] 12 - [OpenShift Installation Configuration]() 13 - [Terraform](#Terraform) 14 - [Pre-Requisites](#pre-requisites-1) 15 - [Build a Cluster](#build-a-cluster-1) 16 17 # PowerShell 18 This section will describe the process to generate the vSphere VMs using PowerShell and the supplied scripts in this module. 19 20 ## Pre-requisites 21 * PowerShell 22 * PowerShell VMware.PowerCLI Module 23 * PowerShell EPS Module 24 25 ## PowerShell Setup 26 27 PowerShell will need to have a couple of plugin installed in order for our script to work. The plugins we need to install are VMware.PowerCLI and EPS. 28 29 ### VMware.PowerCLI 30 31 To install the VMware.PowerCLI, you can run the following command: 32 33 ```shell 34 pwsh -Command 'Install-Module VMware.PowerCLI -Force -Scope CurrentUser' 35 ``` 36 37 ### EPS 38 39 To install the EPS module, you can run the following command: 40 41 ```shell 42 pwsh -Command 'Install-Module -Name EPS -RequiredVersion 1.0 -Force -Scope CurrentUser' 43 ``` 44 45 ### Generating CLI Credentials 46 47 The PowerShell scripts require that a credentials file be generated with the credentials to be used for generating the vSphere resources. This does not have to be the credentials used by the OpenShift cluster via the install-config.yaml, but must have all permissions to create folders, tags, templates, and vms. To generate the credentials files, run: 48 49 ```shell 50 pwsh -command "\$User='<username>';\$Password=ConvertTo-SecureString -String '<password>' -AsPlainText -Force;\$Credential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList \$User, \$Password;\$Credential | Export-Clixml secrets/vcenter-creds.xml" 51 ``` 52 53 Be sure to modify `<username>` to be the username for vCenter and `<password>` to the your password. The output of this needs to go into `secrets/vcenter-creds.xml`. Make sure the secrets directory exists before running the credentials generation command above. 54 55 ## Script Configuration 56 57 The PowerShell script provided by this project provides examples on how to do several aspects to creating a UPI cluster environment. It is configurable to do as much or as little as you need. For the CI build process, we will handle all install-config.yaml configuration, uploading of templates, and monitoring of installation progress. This project can handle doing all that as well if configured appropriately. 58 59 ### Behavioral Configurations 60 61 These properties are used to control how the script works. All of the properties will dictate how much or little work the script will do. 62 63 | Property | Description | 64 |---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| 65 | createInstallConfig | Enable script to create install config based on configuration of variables.ps1. If set to false, the script will not generate install-config.yaml. Note, install-config.yaml will be needed if you want this script to generate the ignition files for vm creation. | 66 | downloadInstaller | Enable script to download installer to be used. If not downloading the installer, the installer must be placed in the same directory as this script. The installer is needed to determine what template to upload to the cluster for vm cloning. | 67 | uploadTemplateOva | Enable script to upload OVA template to be used for all VM being created. | 68 | generateIgnitions | Enable script to generate ignition configs. This is normally used when install-config.yaml is provided to script, but need script to generate the ignition configs for VMs. | 69 | waitForComplete | This option has the script step through the process of waiting for installation complete. Most of this functionality is provided by `openshift-install wait-for`. The script will will check for when api is ready, bootstrap complete, accept CSRs and then for all COs to be done installing. | 70 | delayVMStart | This option has the script delay the start of the VMs after their creation. | 71 72 73 ### vCenter Configuration 74 75 These properties are related to vCenter. These will be used by script to create the install-config.yaml as well as define a single failure domain if no failure domains are configured in the variables.ps1 file. 76 77 | Property | Description | 78 |-------------|------------------------------------------------------------------------------------------------------------------------------------------------| 79 | vcenter | This property specifies the vCenter URL | 80 | username | Specifies the username used to connect to vCenter. This is only used in generating the install-config.yaml information for use by the cluster | 81 | password | The password to be used to connect to vCenter. This is only used in generating the install-config.yaml information for use by the cluster | 82 | portgroup | The default port group to use. This property will be used when no failure domains are defined. | 83 | datastore | The default data store to use. This property will be used when no failure domains are defined. | 84 | datacenter | The default data center to use. This property will be used when no failure domains are defined. | 85 86 87 ### Cluster Configuration 88 89 These properties are used to configure the new cluster. These properties will also be used to determine how to create the VMs. 90 91 | Property | Description | 92 |-----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------| 93 | clustername | The name of the cluster to create. | 94 | basedomain | The base domain to use for the cluster. | 95 | sshkeypath | The path of the ssh key to use when generating ignition configs. | 96 | failure_domains | Contains the failure domains for the cluster. This needs to be a JSON formatted configuration following what is provided in the example below table. | 97 98 Example of a failure domain configuration: 99 100 ```powershell 101 $failure_domains = @" 102 [ 103 { 104 "datacenter": "IBMCloud", 105 "cluster": "vcs-mdcnc-workload-1", 106 "datastore": "mdcnc-ds-1", 107 "network": "ocp-ci-seg-14" 108 },{ 109 "datacenter": "IBMCloud", 110 "cluster": "vcs-mdcnc-workload-2", 111 "datastore": "mdcnc-ds-2", 112 "network": "ocp-ci-seg-14" 113 },{ 114 "datacenter": "IBMCloud", 115 "cluster": "vcs-mdcnc-workload-3", 116 "datastore": "mdcnc-ds-3", 117 "network": "ocp-ci-seg-14" 118 },{ 119 "datacenter": "datacenter-2", 120 "cluster": "vcs-mdcnc-workload-4", 121 "datastore": "mdcnc-ds-4", 122 "network": "ocp-ci-seg-14" 123 } 124 ] 125 "@ 126 ``` 127 128 ### VM Configuration 129 130 | Property | Description | 131 |----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------| 132 | vm_template | Name of the existing vm template in vCenter to use. Use this option to prevent script from uploading a template and use an existing template. | 133 | dns | DNS server for network configuration of all VMs. | 134 | gateway | Gateway for network configuration of all VMs. | 135 | netwask | The network mask for all VMs. | 136 | lb_ip_address | The IP address to use for the load balancer VM. | 137 | bootstrap_ip_address | The IP address to use for the bootstrap VM. | 138 | control_plane_memory | Amount of memory to assign each control plane VM. This value is in MB. | 139 | control_plane_num_cpus | Number of CPUs to assign to each control plane VM. | 140 | control_plane_count | Number of control plane VMs. | 141 | control_plane_ip_addresses | The IP addresses to assign all control plane VMs. | 142 | control_plane_hostnames | The host names to assign to all control plane VMs. | 143 | compute_memory | Amount of memory to assign each compute VM. This value is in MB. | 144 | compute_num_cpus | Number of CPUs to assign to each compute VM. | 145 | compute_count | Number of compute VMs. | 146 | compute_ip_addresses | The IP addresses to assign all compute VMs. | 147 | compute_hostnames | The host names to assign to all compute VMs. | 148 149 150 ## Build a Cluster 151 152 ### Manual Method 153 154 1. Create an install-config.yaml. The machine CIDR for the dev cluster is 139.178.89.192/26. 155 156 ``` 157 apiVersion: v1 158 baseDomain: devcluster.openshift.com 159 metadata: 160 name: mstaeble 161 networking: 162 machineNetwork: 163 - cidr: "139.178.89.192/26" 164 platform: 165 vsphere: 166 vCenter: vcsa.vmware.devcluster.openshift.com 167 username: YOUR_VSPHERE_USER 168 password: YOUR_VSPHERE_PASSWORD 169 datacenter: dc1 170 defaultDatastore: nvme-ds1 171 pullSecret: YOUR_PULL_SECRET 172 sshKey: YOUR_SSH_KEY 173 ``` 174 175 2. Run `openshift-install create manifest`. 176 177 3. Update any configurations you need before generating ignition configs. It is recommended to remove the master machine CR and the worker machineset CR. This can be accomplished by running: 178 179 ``` 180 rm -f openshift/99_openshift-cluster-api_master-machines-*.yaml openshift/99_openshift-cluster-api_worker-machineset-*.yaml 181 ``` 182 183 3. Run `openshift-install create ignition-configs`. 184 185 4. Create and configure a variables.ps1 file. 186 There is an example variables.ps1 file in this directory named variables.ps1.example. The example file contains all properties that are available to be configured. At a minimum, you need to set values for the following variables. 187 * clustername 188 * basedomain 189 * username 190 * password 191 * vcenter 192 * vcentercredpath 193 194 The bootstrap ignition config must be placed in a location that will be accessible by the bootstrap machine. For example, you could store the bootstrap ignition config in a gist. 195 196 Even if declaring static IPs a DHCP server is still required early in the boot process to download the ignition files. 197 198 5. Run `pwsh -f upi.ps1`. 199 200 The script will now attempt to create all VMs based on the config provided. The script will start the VMs and will say `Install Comlete` when all VMs are cloned and started. 201 202 6. Run `openshift-install wait-for install-complete`. Wait for the cluster install to finish. 203 204 7. Enjoy your new OpenShift cluster. 205 206 8. Run `pwsh -f upi-destroy.ps1` to clean up all folder, VMs, and tags. 207 208 # Terraform 209 This section will walk you through generating a cluster using Terraform. 210 211 <a id="terraform-pre-requisites"></a> 212 ## Pre-Requisites 213 214 * terraform 215 * jq 216 217 ## Build a Cluster 218 219 1. Create an install-config.yaml. 220 The machine CIDR for the dev cluster is 139.178.89.192/26. 221 222 ``` 223 apiVersion: v1 224 baseDomain: devcluster.openshift.com 225 metadata: 226 name: mstaeble 227 networking: 228 machineNetwork: 229 - cidr: "139.178.89.192/26" 230 platform: 231 vsphere: 232 vCenter: vcsa.vmware.devcluster.openshift.com 233 username: YOUR_VSPHERE_USER 234 password: YOUR_VSPHERE_PASSWORD 235 datacenter: dc1 236 defaultDatastore: nvme-ds1 237 pullSecret: YOUR_PULL_SECRET 238 sshKey: YOUR_SSH_KEY 239 ``` 240 241 2. Run `openshift-install create ignition-configs`. 242 243 3. Fill out a terraform.tfvars file with the ignition configs generated. 244 There is an example terraform.tfvars file in this directory named terraform.tfvars.example. The example file is set up for use with the dev cluster running at vcsa.vmware.devcluster.openshift.com. At a minimum, you need to set values for the following variables. 245 * cluster_id 246 * cluster_domain 247 * vsphere_user 248 * vsphere_password 249 * ipam_token OR bootstrap_ip, control_plane_ips, and compute_ips 250 251 The bootstrap ignition config must be placed in a location that will be accessible by the bootstrap machine. For example, you could store the bootstrap ignition config in a gist. 252 253 Even if declaring static IPs a DHCP server is still required early in the boot process to download the ignition files. 254 255 4. Run `terraform init`. 256 257 5. Ensure that you have you AWS profile set and a region specified. The installation will use create AWS route53 resources for routing to the OpenShift cluster. 258 259 6. Run `terraform apply -auto-approve`. 260 This will reserve IP addresses for the VMs. 261 262 7. Run `openshift-install wait-for bootstrap-complete`. Wait for the bootstrapping to complete. 263 264 8. Run `terraform apply -auto-approve -var 'bootstrap_complete=true'`. 265 This will destroy the bootstrap VM. 266 267 9. Run `openshift-install wait-for install-complete`. Wait for the cluster install to finish. 268 269 10. Enjoy your new OpenShift cluster. 270 271 11. Run `terraform destroy -auto-approve`.