github.com/pachyderm/pachyderm@v1.13.4/doc/docs/master/contributing/setup.md (about)

     1  # Setup for contributors
     2  
     3  ## General requirements
     4  
     5  First, go through the general [Local Installation Instructions](https://docs.pachyderm.com/latest/getting_started/local_installation/). Additionally, make sure you have the following installed:
     6  
     7  - golang 1.12+
     8  - docker
     9  - [jq](https://stedolan.github.io/jq/)
    10  - [pv](http://ivarch.com/programs/pv.shtml)
    11  - shellcheck
    12  
    13  ## Bash helpers
    14  
    15  To stay up to date, we recommend doing the following.
    16  
    17  First clone the code:
    18  (Note, as of 07/11/19 pachyderm is using go modules and recommends cloning the code outside of the $GOPATH, we use the location ~/workspace as an example, but the code can live anywhere)
    19  
    20      cd ~/workspace
    21      git clone git@github.com:pachyderm/pachyderm
    22  
    23  Then update your `~/.bash_profile` by adding the line:
    24  
    25      source ~/workspace/pachyderm/etc/contributing/bash_helpers
    26  
    27  And you'll stay up to date!
    28  
    29  ## Special macOS configuration
    30  
    31  ### File descriptor limit
    32  
    33  If you're running tests locally, you'll need to up your file descriptor limit. To do this, first setup a LaunchDaemon to up the limit with sudo privileges:
    34  
    35      sudo cp ~/workspace/pachyderm/etc/contributing/com.apple.launchd.limit.plist /Library/LaunchDaemons/
    36  
    37  Once you restart, this will take effect. To see the limits, run:
    38  
    39      launchctl limit maxfiles
    40  
    41  Before the change is in place you'll see something like `256    unlimited`. After the change you'll see a much bigger number in the first field. This ups the system wide limit, but you'll also need to set a per-process limit.
    42  
    43  Second, up the per process limit by adding something like this to your `~/.bash_profile` :
    44  
    45      ulimit -n 12288
    46  
    47  Unfortunately, even after setting that limit it never seems to report the updated version. So if you try
    48  
    49      ulimit
    50  
    51  And just see `unlimited`, don't worry, it took effect.
    52  
    53  To make sure all of these settings are working, you can test that you have the proper setup by running:
    54  
    55      make test-pfs-server
    56  
    57  If this fails with a timeout, you'll probably also see 'too many files' type of errors. If that test passes, you're all good!
    58  
    59  ### Timeout helper
    60  
    61  You'll need the `timeout` utility to run the `make launch` task. To install on mac, do:
    62  
    63      brew install coreutils
    64  
    65  And then make sure to prepend the following to your path:
    66  
    67      PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"
    68  
    69  ## Dev cluster
    70  
    71  Now launch the dev cluster: `make launch-dev-vm`.
    72  
    73  And check it's status: `kubectl get all`.
    74  
    75  ## pachctl
    76  
    77  This will install the dev version of `pachctl`:
    78  
    79  ```
    80      cd ~/workspace/pachyderm
    81      make install
    82      pachctl version
    83  ```
    84  
    85  And make sure that `$GOPATH/bin` is on your `$PATH` somewhere
    86  
    87  ## Getting some images in place for local test runs
    88  
    89  The following commands will put some images that some of the tests rely on in
    90  place in your minikube cluster:
    91  
    92  For `pachyderm_entrypoint` container:
    93  
    94  ```
    95  make docker-build-test-entrypoint
    96  ./etc/kube/push-to-minikube.sh pachyderm_entrypoint
    97  ```
    98  
    99  For `pachyderm/python-build` container:
   100  
   101  ```
   102  (cd etc/pipeline-build; make push-to-minikube)
   103  ```
   104  
   105  ## Running tests
   106  
   107  Now that we have a dev cluster, it's nice to be able to run some tests locally
   108  as we are developing.
   109  
   110  To run some specific tests, just use `go test` directly, e.g:
   111  ```
   112  go test -v ./src/server/cmd/pachctl/cmd
   113  ```
   114  
   115  We don't recommend trying to run all the tests locally, they take a while. Use
   116  CI for that.
   117  
   118  ## Fully resetting
   119  
   120  Instead of running the makefile targets to re-compile `pachctl` and redeploy
   121  a dev cluster, we have a script that you can use to fully reset your pachyderm
   122  environment:
   123  
   124  1) All existing cluster data is deleted
   125  2) If possible, the virtual machine that the cluster is running on is wiped
   126  out
   127  3) `pachctl` is recompiled
   128  4) The dev cluster is re-deployed
   129  
   130  This reset is a bit more time consuming than running one-off Makefile targets,
   131  but comprehensively ensures that the cluster is in its expected state, and is
   132  especially helpful when you're first getting started with contributions and
   133  don't yet have a complete intuition on the various ways a cluster may get in
   134  an unexpected state. It's been tested on docker for mac and minikube, but
   135  likely works in other kubernetes environments as well.
   136  
   137  To run it, simply call `./etc/reset.py` from the pachyderm repo root.