github.com/quite/nomad@v0.8.6/website/source/guides/cluster/requirements.html.md (about)

     1  ---
     2  layout: "guides"
     3  page_title: "Nomad Client and Server Requirements"
     4  sidebar_current: "guides-cluster-requirements"
     5  description: |-
     6    Learn about Nomad client and server requirements such as memory and CPU
     7    recommendations, network topologies, and more.
     8  ---
     9  
    10  # Cluster Requirements
    11  
    12  ## Resources (RAM, CPU, etc.)
    13  
    14  **Nomad servers** may need to be run on large machine instances. We suggest
    15  having between 4-8+ cores, 16-32 GB+ of memory, 40-80 GB+ of disk and
    16  significant network bandwidth. The core count and network recommendations are to
    17  ensure high throughput as Nomad heavily relies on network communication and as
    18  the Servers are managing all the nodes in the region and performing scheduling.
    19  The memory and disk requirements are due to the fact that Nomad stores all state
    20  in memory and will store two snapshots of this data onto disk. Thus disk should
    21  be at least 2 times the memory available to the server when deploying a high
    22  load cluster.
    23  
    24  These recommendations are guidelines and operators should always monitor the
    25  resource usage of Nomad to determine if the machines are under or over-sized.
    26  
    27  **Nomad clients** support reserving resources on the node that should not be
    28  used by Nomad. This should be used to target a specific resource utilization per
    29  node and to reserve resources for applications running outside of Nomad's
    30  supervision such as Consul and the operating system itself.
    31  
    32  Please see the [reservation configuration](/docs/agent/configuration/client.html#reserved) for
    33  more detail.
    34  
    35  ## Network Topology
    36  
    37  **Nomad servers** are expected to have sub 10 millisecond network latencies
    38  between each other to ensure liveness and high throughput scheduling. Nomad
    39  servers can be spread across multiple datacenters if they have low latency
    40  connections between them to achieve high availability.
    41  
    42  For example, on AWS every region comprises of multiple zones which have very low
    43  latency links between them, so every zone can be modeled as a Nomad datacenter
    44  and every Zone can have a single Nomad server which could be connected to form a
    45  quorum and a region.
    46  
    47  Nomad servers uses Raft for state replication and Raft being highly consistent
    48  needs a quorum of servers to function, therefore we recommend running an odd
    49  number of Nomad servers in a region.  Usually running 3-5 servers in a region is
    50  recommended. The cluster can withstand a failure of one server in a cluster of
    51  three servers and two failures in a cluster of five servers. Adding more servers
    52  to the quorum adds more time to replicate state and hence throughput decreases
    53  so we don't recommend having more than seven servers in a region.
    54  
    55  **Nomad clients** do not have the same latency requirements as servers since they
    56  are not participating in Raft. Thus clients can have 100+ millisecond latency to
    57  their servers. This allows having a set of Nomad servers that service clients
    58  that can be spread geographically over a continent or even the world in the case
    59  of having a single "global" region and many datacenter.
    60  
    61  ## Ports Used
    62  
    63  Nomad requires 3 different ports to work properly on servers and 2 on clients,
    64  some on TCP, UDP, or both protocols. Below we document the requirements for each
    65  port.
    66  
    67  * HTTP API (Default 4646). This is used by clients and servers to serve the HTTP
    68    API. TCP only.
    69  
    70  * RPC (Default 4647). This is used by servers and clients to communicate among
    71    each other. TCP only.
    72  
    73  * Serf WAN (Default 4648). This is used by servers to gossip both over the LAN and
    74    WAN to other servers. It isn't required that Nomad clients can reach this address.
    75    TCP and UDP.
    76  
    77  When tasks ask for dynamic ports, they are allocated out of the port range
    78  between 20,000 and 32,000. This is well under the ephemeral port range suggested
    79  by the [IANA](https://en.wikipedia.org/wiki/Ephemeral_port). If your operating
    80  system's default ephemeral port range overlaps with Nomad's dynamic port range,
    81  you should tune the OS to avoid this overlap.
    82  
    83  On Linux this can be checked and set as follows:
    84  
    85  ```
    86  $ cat /proc/sys/net/ipv4/ip_local_port_range 
    87  32768   60999
    88  $ echo "49152 65535" > /proc/sys/net/ipv4/ip_local_port_range 
    89  ```