github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.10.x/troubleshooting/general_troubleshooting.md (about) 1 # General Troubleshooting 2 3 Here are some common issues by symptom along with steps to resolve them. 4 5 - [Connecting to a Pachyderm cluster](#connecting-to-a-pachyderm-cluster) 6 - [Cannot connect via `pachctl` - context deadline exceeded](#cannot-connect-via-pachctl-context-deadline-exceeded) 7 - [Certificate error when using `kubectl`](#certificate-error-when-using-kubectl) 8 - [Uploads and Downloads are slow](#uploads-and-downloads-are-slow) 9 10 11 --- 12 13 ## Connecting to a Pachyderm Cluster 14 15 ### Cannot connect via `pachctl` - context deadline exceeded 16 17 #### Symptom 18 19 You may be using the pachd address config value or environment variable to specify how `pachctl` talks to your Pachyderm cluster, or you may be forwarding the pachyderm port. In any event, you might see something similar to: 20 21 ``` 22 pachctl version 23 COMPONENT VERSION 24 pachctl 1.9.5 25 context deadline exceeded 26 ``` 27 28 Also, you might get this message if `pachd` is not running. 29 30 #### Recourse 31 32 It's possible that the connection is just taking a while. Occasionally this can happen if your cluster is far away (deployed in a region across the country). Check your internet connection. 33 34 It's also possible that you haven't poked a hole in the firewall to access the node on this port. Usually to do that you adjust a security rule (in AWS parlance a security group). For example, on AWS, if you find your node in the web console and click on it, you should see a link to the associated security group. Inspect that group. There should be a way to "add a rule" to the group. You'll want to enable TCP access (ingress) on port 30650. You'll usually be asked which incoming IPs should be whitelisted. You can choose to use your own, or enable it for everyone (0.0.0.0/0). 35 36 37 ### Certificate Error When Using Kubectl 38 39 #### Symptom 40 41 This can happen on any request using `kubectl` (e.g. `kubectl get all`). In particular you'll see: 42 43 ``` 44 kubectl version 45 Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T20:41:24Z", GoVersion:"go1.8.1", Compiler:"gc", Platform:"darwin/amd64"} 46 Unable to connect to the server: x509: certificate signed by unknown authority 47 ``` 48 49 #### Recourse 50 51 Check if you're on any sort of VPN or other egress proxy that would break SSL. Also, there is a possibility that your credentials have expired. In the case where you're using GKE and gcloud, renew your credentials via: 52 53 ``` 54 kubectl get all 55 Unable to connect to the server: x509: certificate signed by unknown authority 56 ``` 57 58 ``` 59 gcloud container clusters get-credentials my-cluster-name-dev 60 Fetching cluster endpoint and auth data. 61 kubeconfig entry generated for my-cluster-name-dev. 62 ``` 63 64 ``` 65 kubectl config current-context 66 gke_my-org_us-east1-b_my-cluster-name-dev 67 ``` 68 69 ### Uploads and Downloads are Slow 70 71 #### Symptom 72 73 Any `pachctl put file` or `pachctl get file` commands are slow. 74 75 #### Recourse 76 77 If you do not explicitly set the pachd address config value, `pachctl` will default to using port forwarding, which throttles traffic to ~1MB/s. If you need to do large downloads/uploads you should consider using pachd address config value. You'll also want to make sure you've allowed ingress access through any firewalls to your k8s cluster. 78 79 ### Naming a Repo with an Unsupported Symbol 80 81 #### Symptom 82 83 A Pachyderm repo was accidentally named starting with a dash (`-`) and the repository 84 is treated as a command flag instead of a repository. 85 86 #### Recourse 87 88 Pachyderm supports standard `bash` utilities that you can 89 use to resolve this and similar problems. For example, in this case, 90 you can specify double dashes (`--`) to delete the repository. Double dashes 91 signify the end of options and tell the shell to process the 92 rest arguments as filenames and objects. 93 94 For more information, see `man bash`.