github.com/openshift/installer@v1.4.17/docs/user/azure/limits.md (about)

     1  # Limits
     2  
     3  You can find a comprehensive list of the default Azure service limits published here:
     4  
     5  [Azure Service Limits][service-limits]
     6  
     7  Below, we'll identify OpenShift cluster needs and how those impact some of those limits.
     8  
     9  ## VNet
    10  
    11  Each cluster creates its own VNet. The default limit of VNets per regions is 1000 and will allow 1000 clusters. To 
    12  have more than 1000 clusters, you will need to increase this limit. The default installation creates a single VNet.
    13  
    14  ## Network Interfaces
    15  
    16  The default installation creates 6 network interfaces. The default limit per region is 65536. Additional network interfaces are
    17  created for additional machines and load balancers created by cluster usage and deployed workloads.
    18  
    19  ## Public and Private IP Addresses
    20  
    21  By default, the installer distributes control-plane and compute machines across [all availability zones within a region][availability-zones] to provision the cluster in a highly available configuration.
    22  Please see [this map][az-map] for a current region map with availability zone count.
    23  We recommend selecting regions with 3 or more availability zones.
    24  You can [provide an install-config](../overview.md#multiple-invocations) to [configure](customization.md) the installer to use specific zones to override the defaults.
    25  
    26  The installer creates two external load balancers and one internal load balancer. The external load balancers each have a public IP address while the internal load balancer has a private IP address. Two subnets are created within the VNet. One of the subnets is for the control-plane nodes while the other subnet is for the compute nodes. A description of what each load balancer does is [here](#load-balancing).
    27  
    28  Each VM gets a private IP address. The default install creates 6 VM's for a total of 6 private addresses. The internal load balancer has a private IP address while the external load balancers have public IP addresses. This sums up to 7 private IP addresses and 2 public IP addresses.
    29  
    30  A public IP address is also created for the bootstrap machine during installation. This is so that it is available via SSH should anything go wrong during the install. The bootstrap machine and public IP address are destroyed once installation is complete.
    31  
    32  ## Network Security Groups
    33  
    34  Each cluster creates network security groups for every subnet within the VNet. The default install creates network
    35  security groups for the control plane and for the compute node subnets. The default limit of 5000 for new accounts
    36  allows for many clusters to be created. The network security groups which exist after the default install are:
    37  
    38  1. controlplane
    39     * This allows the control-plane to be reached on port 6443 from anywhere
    40  2. node
    41     * This allows worker nodes to be reached from the internet on ports 80 and 443
    42  
    43  ## Instance Limits
    44  
    45  By default, an x86 cluster will create:
    46  
    47  * One Standard_D4s_v3 bootstrap machine (removed after install)
    48  * Three Standard_D8s_v3 master nodes.
    49  * Three Standard_D2s_v3 worker nodes.
    50  
    51  The specs for the VM sizes (Dsv3-series) are as follows:
    52  
    53  * Standard_D8s_v3
    54     * 8 vCPU's, 32GiB ram
    55     * IOPs / Throughput (Mbps): (cached) 16000 / 128
    56     * IOPs / Throughput (Mbps): (uncached) 12800 / 192
    57     * NICs / Bandwidth (Mbps): 4 / 4000
    58     * 64 GiB temp storage (SSD)
    59     * 16 data disks max
    60  
    61  * Standard_D4s_v3
    62     * 4 vCPU's, 16GiB ram
    63     * IOPs / Throughput (Mbps): (cached) 8000 / 512
    64     * IOPs / Throughput (Mbps): (uncached) 6400 / 1152
    65     * NICs / Bandwidth (Mbps): 2 / 2000
    66     * NICs / Bandwidth (Mbps): 2 / 1000
    67     * 32 GiB temp storage (SSD)
    68     * 8 data disks max
    69  
    70  * Standard_D2s_v3
    71     * 2 vCPU's, 8GiB ram
    72     * IOPs / Throughput (Mbps): (cached) 4000 / 256
    73     * IOPs / Throughput (Mbps): (uncached) 3200 / 384
    74     * NICs / Bandwidth (Mbps): 2 / 1000
    75     * 16 GiB temp storage (SSD)
    76     * 4 data disks max
    77  
    78  More details on VM sizes can be found [here][sizes-general].
    79  
    80  By default, an arm64 cluster will create:
    81  
    82  * One Standard_D8ps_v5 bootstrap machine (removed after install)
    83  * Three Standard_D8ps_v5 master nodes.
    84  * Three Standard_D4ps_v5 worker nodes.
    85  
    86  The specs for the VM sizes (Dpsv5-series) are as follows:
    87  
    88  * Standard_D8ps_v5
    89     * 8 vCPU's, 32GiB ram
    90     * IOPs / Throughput (Mbps): (uncached burst) 20000 / 1200
    91     * IOPs / Throughput (Mbps): (uncached) 12800 / 290
    92     * NICs / Bandwidth (Mbps): 4 / 12500
    93     * 16 data disks max
    94     * Remote Storage Only
    95  
    96  * Standard_D4ps_v5
    97     * 4 vCPU's, 16GiB ram
    98     * IOPs / Throughput (Mbps): (uncached burst) 20000 / 1200
    99     * IOPs / Throughput (Mbps): (uncached) 6400 / 145
   100     * NICs / Bandwidth (Mbps): 2 / 12500
   101     * 8 data disks max
   102     * Remote Storage Only
   103  
   104  More details on VM sizes can be found [here][sizes-arm64]. All VMs are Gen2 only.
   105  
   106  The default subscription only allows for 20 vCPU's and will need to be [increased](#increasing-limits) to at least 22.
   107  If you intend to start with a higher number of workers, enable autoscaling and large workloads
   108  or a different instance type, please ensure you have the necessary remaining instance count within the instance type's
   109  limit to satisfy the need. If not, please ask Azure to increase the limit via a support case.
   110  
   111  
   112  ## Load Balancing
   113  
   114  By default, each cluster will create 3 network load balancers. The default limit per region is 1000. The following load balancers are created:
   115  
   116  1. default 
   117    * Public IP address that load balances requests to ports 80 and 443 across worker nodes
   118  2. internal
   119    * Private IP address that load balances requests to ports 6443 and 22623 across control-plane nodes
   120  3. external
   121    * Public IP address that load balances requests to port 6443 across control-plane nodes
   122  
   123  Additional Kubernetes LoadBalancer Service objects will create additional [load balancers][load-balancing]. 
   124  
   125  
   126  ## Increasing limits
   127  
   128  
   129  To increase a limit beyond the maximum, a support request will need to be filed.
   130  
   131  First, click on "help + support". It is located on the bottom left menu.
   132  
   133  ![](images/limits_Microsoft_Azure_0.png)
   134  
   135  Next, click on "New support request"
   136  
   137  ![](images/limits_Microsoft_Azure_1.png)
   138  
   139  From here, you'll want to specify the issue type of "Service and subscription limits (quotas)"
   140  
   141  ![](images/limits_Microsoft_Azure_2.png)
   142  
   143  Pick the subscription you'll be updating as well as the quota type.
   144  
   145  ![](images/limits_Microsoft_Azure_3.png)
   146  ![](images/limits_Microsoft_Azure_4.png)
   147  
   148  Once you've specified the subscription and quota type, you'll need to fill out your contact information.
   149  
   150  ![](images/limits_Microsoft_Azure_5.png)
   151  
   152  
   153  You will then review and submit your request.
   154  
   155  ![](images/limits_Microsoft_Azure_7.png)
   156  
   157  
   158  [availability-zones]: https://azure.microsoft.com/en-us/global-infrastructure/availability-zones/
   159  [az-map]: https://azure.microsoft.com/en-us/global-infrastructure/regions/
   160  
   161  [network-ip]: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-ip-addresses-overview-arm
   162  
   163  [load-balancing]: https://docs.microsoft.com/en-us/azure/load-balancer/load-balancer-overview
   164  [service-limits]: https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits
   165  [sizes-general]: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes-general
   166  [sizes-arm64]: https://docs.microsoft.com/en-us/azure/virtual-machines/dpsv5-dpdsv5-series