github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/how-tos/export-data-out-pachyderm/mount-repo-to-local-computer.md (about)

     1  # Mount a Repo to a Local Computer
     2  
     3  !!! warning
     4      Pachyderm uses FUSE to mount repositories as local filesystems.
     5      Because Apple has announced phasing out support for macOS
     6      kernel extensions, including FUSE, this functionality is no
     7      longer stable on macOS Catalina (10.15) or later.
     8  
     9  Pachyderm enables you to mount a repository
    10  as a local filesystem on your computer by using the
    11  `pachctl mount` command. This command
    12  uses the Filesystem in Userspace (FUSE) user interface to export a Pachyderm
    13  File System (PFS) to a Unix computer system.
    14  This functionality is useful when you want to pull data locally to experiment,
    15  review the results of a pipeline, or modify the files
    16  in the input repository directly.
    17  
    18  You can mount a Pachyderm repo in one of the following modes:
    19  
    20  * `Read-only` — you can read the mounted files to further experiment with them
    21  locally, but cannot modify them.
    22  * `Read-write` — you can read mounted files, modify their contents, and
    23  push them back into your centralized Pachyderm input repositories.
    24  
    25  ## Mounting Repositories in Read-Only Mode
    26  
    27  By default, when you run the `pachctl mount` command, Pachyderm mounts
    28  all repositories in read-only mode. You can access the
    29  files through your file browser or enable third-party applications
    30  access. Read-only access enables you to explore and experiment with
    31  the data, without modifying it. For example, you can mount your
    32  repo to a local computer and then open that directory in a Jupyter
    33  Notebook for exploration. 
    34  
    35  ## Mounting Specific Branches and Commits
    36  
    37  The `pachctl mount` command allows you to mount not only the default
    38  branch, typically a `master` branch, but also other Pachyderm
    39  branches. By default, Pachyderm mounts the `master` branch. However,
    40  if you add a branch to the name of the repo, the `HEAD` of that branch
    41  will be mounted.
    42  
    43  **Example:**
    44  
    45  ```shell
    46  pachctl mount images --repos images@staging
    47  ```
    48  
    49  Also, you can mount a specific commit, but because commits
    50  might be on multiple branches, modifying them might result in data deletion
    51  in the `HEAD` of the branches. Therefore, you can only mount commits in
    52  read-only mode. If you want to write to a specific commit that is not
    53  the `HEAD` of a branch, you can create a new branch with that commit as `HEAD`.
    54  
    55  ## Mounting Repositories in Read-Write Mode
    56  
    57  Running the `pachctl mount` command with the `--write` flag grants you
    58  write access to the mounted repositories, which means that you can
    59  open the files for editing and put them back to the Pachyderm
    60  repository. 
    61  
    62  For example, you have the [OpenCV example pipeline](../../../getting_started/beginner_tutorial/#image-processing-with-opencv)
    63  up and running. If you want to edit files in the `images`
    64  repository, experiment with brightness and contrast
    65  settings in `liberty.png`, and finally have your `edges`
    66  pipeline process those changes.
    67  If you do not mount the `images` repo, you would have to
    68  first download the files to your computer, edit them,
    69  and then put them back to the repository. The `pachctl mount`
    70  command automates all these steps for you. You can mount just the
    71  `images` repo or all Pachyderm repositories as directories
    72  on you machine, edit as needed, and, when done,
    73  exit the `pachctl mount` command. Upon exiting the `pachctl mount`
    74  command, Pachyderm uploads all the changes to the corresponding
    75  repository.
    76  
    77  If someone else modifies the files while you are working on them
    78  locally, their changes will likely be overwritten when you exit
    79  `pachctl mount`. This happens because your changes are saved to
    80  the Pachyderm repository only after you interrupt the `pachctl mount`
    81  command. Therefore, make sure that you do not work on the
    82  same files while someone else is working on them.
    83  
    84  !!! warning
    85      Use writable mount **ONLY** when you have sole ownership
    86      over the mounted data. Otherwise, merge conflicts or
    87      unexpected data overwrites can occur.
    88  
    89  Because output repositories are created by the Pachyderm
    90  pipelines, they are immutable. Only a pipeline
    91  can change and update files in these repositories. If you try to change
    92  a file in an output repo, you will get an error message.
    93  
    94  ## Prerequisites
    95  
    96  You must have the following configured for this functionality to work:
    97  
    98  * Unix or Unix-like operating system, such as Ubuntu 16.04 or macOS
    99  Yosemite or later.
   100  * FUSE for your operating system installed:
   101  
   102    * On macOS, run:
   103  
   104      ```shell
   105      brew cask install osxfuse
   106      ```
   107  
   108    * On Ubuntu, run:
   109  
   110      ```shell
   111      sudo apt-get install -y fuse
   112      ```
   113  
   114      For more information, see:
   115  
   116      * [FUSE for macOS](https://osxfuse.github.io/)
   117  
   118  ## Mount a Pachyderm Repo
   119  
   120  Before you can mount a Pachyderm repo, verify that you have all the
   121  [Prerequisites](#prerequisites).
   122  
   123  To mount a Pachyderm repo on a local computer, complete the following
   124  steps:
   125  
   126  1. In a terminal, go to a directory in which you want to mount a
   127  Pachyderm repo. It can be any new empty directory on your local computer.
   128  For example, `pfs`.
   129  
   130  1. Run `pachctl mount` for a repository and branch that you want to mount:
   131  
   132     ```shell
   133     pachctl mount <path-on-your-computer> [flags]
   134     ```
   135  
   136     **Example:**
   137  
   138     * If you want to mount all the repositories in your Pachyderm cluster 
   139     to a `pfs` directory on your computer and give `WRITE` access to them, run:
   140  
   141     ```shell
   142     pachctl mount pfs --write
   143     ```
   144  
   145     * If you want to mount the master branch of the `images` repo
   146     and enable file editing in this repository, run:
   147  
   148     ```shell
   149     pachctl mount pfs --repos images@master+w
   150     ```
   151  
   152     To give read-only access, omit `+w`.
   153  
   154     **System Response:**
   155  
   156     ```
   157     ro for images: &{Branch:master Write:true}
   158     ri: repo:<name:"montage" > created:<seconds:1591812554 nanos:348079652 > size_bytes:1345398 description:"Output repo for pipeline montage." branches:<repo:<name:"montage" > name:"master" >
   159     continue
   160     ri: repo:<name:"edges" > created:<seconds:1591812554 nanos:201592492 > size_bytes:136795 description:"Output repo for pipeline edges." branches:<repo:<name:"edges" > name:"master" >
   161     continue
   162     ri: repo:<name:"images" > created:<seconds:1591812554 nanos:28450609 > size_bytes:244068 branches:<repo:<name:"images" > name:"master" >
   163     MkdirAll /var/folders/jl/mm3wrxqd75l9r1_d0zktphdw0000gn/T/pfs201409498/images
   164     ```
   165  
   166     The command runs in your terminal until you terminate it
   167     by pressing `CTRL+C`.
   168  
   169  1. You can check that the repo was mounted by running the mount command
   170  in your terminal:
   171  
   172     ```shell hl_lines="7"
   173     mount
   174     /dev/disk1s1 on / (apfs, local, read-only, journaled)
   175     devfs on /dev (devfs, local, nobrowse)
   176     /dev/disk1s2 on /System/Volumes/Data (apfs, local, journaled, nobrowse)
   177     /dev/disk1s5 on /private/var/vm (apfs, local, journaled, nobrowse)
   178     map auto_home on /System/Volumes/Data/home (autofs, automounted, nobrowse)
   179     pachctl@osxfuse0 on /Users/testuser/pfs (osxfuse, nodev, nosuid, synchronous, mounted by testuser)
   180     ```
   181  
   182  1. Access your mountpoint.
   183  
   184     For example, in macOS, open Finder, press
   185     `CMD + SHIFT + G`, and type the mountpoint location. If you have mounted
   186     the repo to `~/pfs`, type `~/pfs`.
   187  
   188     ![finder-repo-mount](../../assets/images/s_finder_repo_mount.png)
   189  
   190  1. Edit the files as needed.
   191  1. When ready, add your changes to the Pachyderm repo by stopping
   192  the `pachctl mount` command with `CTRL+C` or by running `pachctl unmount
   193  <mountpoint>`.
   194  
   195     If you have mounted a writable Pachyderm share interrupting the
   196     `pachctl mount` command results in uploads
   197     your changes to the corresponding repo and branch, which is equivalent
   198     to running the `pachctl put file` command. You can check that
   199     Pachyderm runs a new job for this work by listing current jobs with
   200     `pachctl list job`.