github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/deploy-manage/deploy/connect-to-cluster.md (about) 1 # Connect to a Pachyderm cluster 2 3 After you deploy a Pachyderm cluster, you can continue to use 4 the command-line interface, connect to the Pachyderm dashboard, or 5 configure third-party applications to access your cluster programmatically. 6 Often all you need to do is just continue to use the command-line 7 interface to create repositories, pipelines, and upload your data. 8 At other times, you might have multiple Pachyderm clusters deployed 9 and need to switch between them to perform management operations. 10 11 You do not need to configure anything specific to start using the 12 Pachyderm CLI right after deployment. However, the Pachyderm 13 dashboard and the S3 gateway require 14 explicit port-forwarding or direct access through an externally 15 exposed IP address and port. 16 17 This section describes the various options available for you to connect 18 to your Pachyderm cluster. 19 20 ## Connect to a Local Cluster 21 22 If you are just exploring Pachyderm, use port-forwarding to 23 access both `pachd` and the Pachyderm dashboard. 24 25 By default, Pachyderm enables port-forwarding from `pachctl` to `pachd`. 26 If you do not want to use port-forwarding for `pachctl` operations, 27 configure a `pachd_address` as described in 28 [Connect by using a Pachyderm context](#connect-by-using-a-pachyderm-context). 29 30 To connect to the Pachyderm dashboard, you can either use port-forwarding, 31 or the IP address of the virtual machine on which your Kubernetes cluster 32 is running. 33 34 The following example shows how to access a Pachyderm cluster 35 that runs in `minikube`. 36 37 To connect to a Pachyderm dashboard, complete the following steps: 38 39 * To use port-forwarding, run: 40 41 ```shell 42 pachctl port-forward 43 ``` 44 45 * To use the IP address of the VM:` 46 47 1. Get the minikube IP address: 48 49 ```shell 50 minikube ip 51 ``` 52 53 1. Point your browser to the following address: 54 55 ```shell 56 <minikube_ip>:30080 57 ``` 58 59 ## Connect to a Cluster Deployed on a Cloud Platform 60 61 As in a local cluster deployment, a Pachyderm cluster 62 deployed on a cloud platform has impllcit port-forwarding enabled. 63 This means that, if you are connected to the cluster so 64 that `kubectl` works, `pachctl` can communicate with `pachd` 65 without any additional configuration. 66 Other services still need explicit port forwarding. 67 For example, to access the Pachyderm dashboard, 68 you need to run pachctl port-forward. 69 Since a Pachyderm cluster 70 deployed on a cloud platform is more likely to become 71 a production deployment, configuring a `pachd_address` 72 as described in 73 [Connect by using a Pachyderm context](#connect-by-using-a-pachyderm-context) 74 is the preferred way. 75 76 ## Connect by using a Pachyderm context 77 78 You can specify an IP address that you use to connect to the 79 Pachyderm UI and the S3 gateway by storing that address in the 80 Pachyderm configuration file as the `pachd_address` parameter. 81 If you have already deployed a Pachyderm cluster, you can 82 set a Pachyderm IP address by updating your cluster configuration 83 file. 84 85 This configuration is supported for deployments that do not have 86 a firewall set up between the Pachyderm cluster deployed in the cloud 87 and the client machine. 88 Defining a dedicated `pachd` IP address and host is a more reliable 89 way that might also result in better performance compared to 90 port-forwarding. Therefore, Pachyderm 91 recommends that you use contexts in all production environments. 92 93 This configuration requires that you deploy an ingress controller 94 and a reliable security method to protect the traffic between the 95 Pachyderm pods and your client machine. Remember that exposing your 96 traffic through a public ingress might 97 create a security issue. Therefore, if you expose your `pachd` endpoint, 98 you need to make sure that you take steps to protect the endpoint and 99 traffic against common container security threats. Port-forwarding 100 might be an alternative which might result in sufficient performance 101 if you place the data that is consumed by your pipeline in object 102 store buckets located in the same region. 103 104 For more information about Pachyderm contexts, see 105 [Manage Cluster Access](../manage/cluster-access.md). 106 107 To connect by using a Pachyderm context, complete the following 108 steps: 109 110 1. Get the current context: 111 112 ```shell 113 pachctl config get active-context 114 ``` 115 116 This command returns the name of the currently active context. 117 Use this as the argument to the command below. 118 119 If no IP address is set up for this cluster, you get the following 120 output: 121 122 ```shell 123 pachctl config get context <name> 124 { 125 126 } 127 ``` 128 129 1. Set up `pachd_address`: 130 131 ```shell 132 pachctl config update context <name> --pachd-address <host:port> 133 ``` 134 135 **Example:** 136 137 ```shell 138 pachctl config update context local --pachd-address 192.168.1.15:30650 139 ``` 140 141 **Note:** By default, the `pachd` port is `30650`. 142 143 1. Verify that the configuration has been updated: 144 145 ```shell 146 pachctl config get context local 147 ``` 148 149 **System Response:** 150 151 ```shell 152 { 153 "pachd_address": "192.168.1.15:30650" 154 } 155 ``` 156 157 ## Connect by Using Port-Forwarding 158 159 The Pachyderm port-forwarding is the simplest way to enable cluster access 160 and test basic Pachyderm functionality. Pachyderm automatically starts 161 port-forwarding from `pachctl` to `pachd`. Therefore, the traffic 162 from the local machine goes to the `pachd` endpoint through the 163 Kubernetes API. However, to open a persistent tunnel to other ports, including 164 the Pachyderm dashboard, git and authentication hooks, the built-in HTTP 165 file API, and other, you need to run port-forwarding explicitly. 166 167 Also, if you are connecting with port-forward, you are using the `0.0.0.0`. 168 Therefore, if you are using a proxy, it needs to handle that appropriately. 169 170 Although port-forwarding is convenient for testing, for production 171 environments, this connection might be too slow and unreliable. 172 The speed of the port-forwarded traffic is limited to `1 MB/s`. 173 Therefore, if you experience high latency with `put file` and 174 `get file` operations, or if you anticipate high throughput 175 in your Pachyderm environment, you need to enable ingress access 176 to your Kubernetes cluster. Each cloud provider has its own 177 requirements and procedures for enabling ingress controller and 178 load balancing traffic at the application layer for better performance. 179 180 Remember that exposing your traffic through a public ingress might 181 create a security issue. Therefore, if you expose your `pachd` endpoint, 182 you need to make that you take steps to protect the endpoint and 183 traffic against common container security threats. Port-forwarding 184 might be an alternative which might provide satisfactory performance 185 if you place the data that is consumed by your pipeline in object 186 store buckets located in the same region. 187 188 <!--Add a link to the section that describes the above--> 189 190 To enable port-forwarding, complete the following steps: 191 192 1. Open a new terminal window. 193 1. Run: 194 195 ```shell 196 pachctl port-forward 197 ``` 198 199 This command does not stop unless you manually interrupt it. 200 You can run other `pachctl` commands from another window. 201 If any of your `pachctl` commands hang, verify if the 202 `kubectl` port-forwarding has had issues that prevent 203 `pachctl port-forward` from running properly.