github.com/pachyderm/pachyderm@v1.13.4/examples/opencv/demo-script.md (about)

     1  
     2  Setup
     3  -----
     4  
     5  - Run `./setup-demo.sh`
     6  - You'll want to do that, and run through this script before any demo (to cache the opencv image)
     7  - Then do a `pachctl delete all` and re-create the images repo
     8  
     9  Demo
    10  ----
    11  
    12  ## Exposition:
    13  
    14  Local setup
    15      - minikube
    16      - local VM running k8s
    17  
    18  kubectl get all
    19      - running all on my machine
    20      - from a VM
    21  pachctl list repo
    22      - see images repo
    23      - version control data
    24      - all terms from git
    25  
    26  ## Step 1 -- Add a file
    27  
    28  ```shell
    29  pachctl put file images@master -i examples/opencv/images.txt
    30  ```
    31  
    32      - show them the mount
    33      - locally view the data
    34      - can see repo images
    35      - now there is an image in it
    36      - because everything is snapshotted (commit 0)
    37      - statue of liberty
    38  
    39  ## Step 2 -- Add new images
    40     
    41  ```shell
    42  pachctl put file images@master -i examples/opencv/images2.txt
    43  ```
    44  
    45      - refresh
    46      - new commit -- master/1
    47      - overlays data based on diffs / analgous to git diffs
    48      - first one is there
    49      - two more on top of it
    50  
    51  ## Step 3 -- Run a pipeline
    52  
    53  ```shell
    54  pachctl create pipeline -f edges.json
    55  pachctl list job
    56  ```
    57  
    58      - the pipeline uses openCV
    59      - show powerpoint from dropbox:
    60          - show them the pipeline code (or show in vim)
    61          - take intput / output repo
    62          - multiple inputs
    63          - subscribe to new data coming in on data repo
    64          - describe w json manifest
    65      - dont know opencv
    66          - just pull in a library as a processing step
    67          - plug and play different approaches / steps of analysis
    68      - NOTE!!! create pipeline will take 5 min now if you haven't cached the opencv image
    69          - so do it once before, then do delete all
    70      - see output of pipeline
    71      - output commits correspond to input structure
    72  
    73  ## Step 4 -- add more data
    74  
    75  ```shell
    76  pachctl put file images@master -c -i examples/opencv/images3.txt
    77  ```
    78      - kicks off the pipeline
    79      - see new commit in the edges repo
    80  
    81  Common questions
    82  ---
    83  
    84  - provenance
    85    - look at input commits for one of the outputs
    86  - no reprocessing!
    87    - only processed the new images per commit
    88  - if you delete minikube has to pull container again
    89    - you can just kill pachyderm, thats ok
    90  
    91  Gotchas
    92  ---
    93  
    94  VM network req
    95  broken putfile / do delete all to recover
    96  
    97