github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/how-tos/export-data-out-pachyderm/mount-repo-to-local-computer.md (about) 1 # Mount a Repo to a Local Computer 2 3 !!! warning 4 Pachyderm uses FUSE to mount repositories as local filesystems. 5 Because Apple has announced phasing out support for macOS 6 kernel extensions, including FUSE, this functionality is no 7 longer stable on macOS Catalina (10.15) or later. 8 9 Pachyderm enables you to mount a repository 10 as a local filesystem on your computer by using the 11 `pachctl mount` command. This command 12 uses the Filesystem in Userspace (FUSE) user interface to export a Pachyderm 13 File System (PFS) to a Unix computer system. 14 This functionality is useful when you want to pull data locally to experiment, 15 review the results of a pipeline, or modify the files 16 in the input repository directly. 17 18 You can mount a Pachyderm repo in one of the following modes: 19 20 * `Read-only` — you can read the mounted files to further experiment with them 21 locally, but cannot modify them. 22 * `Read-write` — you can read mounted files, modify their contents, and 23 push them back into your centralized Pachyderm input repositories. 24 25 ## Mounting Repositories in Read-Only Mode 26 27 By default, when you run the `pachctl mount` command, Pachyderm mounts 28 all repositories in read-only mode. You can access the 29 files through your file browser or enable third-party applications 30 access. Read-only access enables you to explore and experiment with 31 the data, without modifying it. For example, you can mount your 32 repo to a local computer and then open that directory in a Jupyter 33 Notebook for exploration. 34 35 ## Mounting Specific Branches and Commits 36 37 The `pachctl mount` command allows you to mount not only the default 38 branch, typically a `master` branch, but also other Pachyderm 39 branches. By default, Pachyderm mounts the `master` branch. However, 40 if you add a branch to the name of the repo, the `HEAD` of that branch 41 will be mounted. 42 43 **Example:** 44 45 ```shell 46 pachctl mount images --repos images@staging 47 ``` 48 49 Also, you can mount a specific commit, but because commits 50 might be on multiple branches, modifying them might result in data deletion 51 in the `HEAD` of the branches. Therefore, you can only mount commits in 52 read-only mode. If you want to write to a specific commit that is not 53 the `HEAD` of a branch, you can create a new branch with that commit as `HEAD`. 54 55 ## Mounting Repositories in Read-Write Mode 56 57 Running the `pachctl mount` command with the `--write` flag grants you 58 write access to the mounted repositories, which means that you can 59 open the files for editing and put them back to the Pachyderm 60 repository. 61 62 For example, you have the [OpenCV example pipeline](../../../getting_started/beginner_tutorial/#image-processing-with-opencv) 63 up and running. If you want to edit files in the `images` 64 repository, experiment with brightness and contrast 65 settings in `liberty.png`, and finally have your `edges` 66 pipeline process those changes. 67 If you do not mount the `images` repo, you would have to 68 first download the files to your computer, edit them, 69 and then put them back to the repository. The `pachctl mount` 70 command automates all these steps for you. You can mount just the 71 `images` repo or all Pachyderm repositories as directories 72 on you machine, edit as needed, and, when done, 73 exit the `pachctl mount` command. Upon exiting the `pachctl mount` 74 command, Pachyderm uploads all the changes to the corresponding 75 repository. 76 77 If someone else modifies the files while you are working on them 78 locally, their changes will likely be overwritten when you exit 79 `pachctl mount`. This happens because your changes are saved to 80 the Pachyderm repository only after you interrupt the `pachctl mount` 81 command. Therefore, make sure that you do not work on the 82 same files while someone else is working on them. 83 84 !!! warning 85 Use writable mount **ONLY** when you have sole ownership 86 over the mounted data. Otherwise, merge conflicts or 87 unexpected data overwrites can occur. 88 89 Because output repositories are created by the Pachyderm 90 pipelines, they are immutable. Only a pipeline 91 can change and update files in these repositories. If you try to change 92 a file in an output repo, you will get an error message. 93 94 ## Prerequisites 95 96 You must have the following configured for this functionality to work: 97 98 * Unix or Unix-like operating system, such as Ubuntu 16.04 or macOS 99 Yosemite or later. 100 * FUSE for your operating system installed: 101 102 * On macOS, run: 103 104 ```shell 105 brew cask install osxfuse 106 ``` 107 108 * On Ubuntu, run: 109 110 ```shell 111 sudo apt-get install -y fuse 112 ``` 113 114 For more information, see: 115 116 * [FUSE for macOS](https://osxfuse.github.io/) 117 118 ## Mount a Pachyderm Repo 119 120 Before you can mount a Pachyderm repo, verify that you have all the 121 [Prerequisites](#prerequisites). 122 123 To mount a Pachyderm repo on a local computer, complete the following 124 steps: 125 126 1. In a terminal, go to a directory in which you want to mount a 127 Pachyderm repo. It can be any new empty directory on your local computer. 128 For example, `pfs`. 129 130 1. Run `pachctl mount` for a repository and branch that you want to mount: 131 132 ```shell 133 pachctl mount <path-on-your-computer> [flags] 134 ``` 135 136 **Example:** 137 138 * If you want to mount all the repositories in your Pachyderm cluster 139 to a `pfs` directory on your computer and give `WRITE` access to them, run: 140 141 ```shell 142 pachctl mount pfs --write 143 ``` 144 145 * If you want to mount the master branch of the `images` repo 146 and enable file editing in this repository, run: 147 148 ```shell 149 pachctl mount pfs --repos images@master+w 150 ``` 151 152 To give read-only access, omit `+w`. 153 154 **System Response:** 155 156 ``` 157 ro for images: &{Branch:master Write:true} 158 ri: repo:<name:"montage" > created:<seconds:1591812554 nanos:348079652 > size_bytes:1345398 description:"Output repo for pipeline montage." branches:<repo:<name:"montage" > name:"master" > 159 continue 160 ri: repo:<name:"edges" > created:<seconds:1591812554 nanos:201592492 > size_bytes:136795 description:"Output repo for pipeline edges." branches:<repo:<name:"edges" > name:"master" > 161 continue 162 ri: repo:<name:"images" > created:<seconds:1591812554 nanos:28450609 > size_bytes:244068 branches:<repo:<name:"images" > name:"master" > 163 MkdirAll /var/folders/jl/mm3wrxqd75l9r1_d0zktphdw0000gn/T/pfs201409498/images 164 ``` 165 166 The command runs in your terminal until you terminate it 167 by pressing `CTRL+C`. 168 169 1. You can check that the repo was mounted by running the mount command 170 in your terminal: 171 172 ```shell hl_lines="7" 173 mount 174 /dev/disk1s1 on / (apfs, local, read-only, journaled) 175 devfs on /dev (devfs, local, nobrowse) 176 /dev/disk1s2 on /System/Volumes/Data (apfs, local, journaled, nobrowse) 177 /dev/disk1s5 on /private/var/vm (apfs, local, journaled, nobrowse) 178 map auto_home on /System/Volumes/Data/home (autofs, automounted, nobrowse) 179 pachctl@osxfuse0 on /Users/testuser/pfs (osxfuse, nodev, nosuid, synchronous, mounted by testuser) 180 ``` 181 182 1. Access your mountpoint. 183 184 For example, in macOS, open Finder, press 185 `CMD + SHIFT + G`, and type the mountpoint location. If you have mounted 186 the repo to `~/pfs`, type `~/pfs`. 187 188  189 190 1. Edit the files as needed. 191 1. When ready, add your changes to the Pachyderm repo by stopping 192 the `pachctl mount` command with `CTRL+C` or by running `pachctl unmount 193 <mountpoint>`. 194 195 If you have mounted a writable Pachyderm share interrupting the 196 `pachctl mount` command results in uploads 197 your changes to the corresponding repo and branch, which is equivalent 198 to running the `pachctl put file` command. You can check that 199 Pachyderm runs a new job for this work by listing current jobs with 200 `pachctl list job`.